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1.  Personal  Background  and  Qualifications 

I  am  a  Professor  in  the  Department  of  Agricultural  &  Resource  Economics  at  the 
University  of  California,  Berkeley.  I  received  a  B.A.  in  Philosophy,  Politics  and  Economics  at 
Oxford  University  in  1965,  an  M.Sc(Econ)  in  Economics  from  the  London  School  of  Economics 
in  1967,  and  a  Ph.D  in  economics  from  Harvard  University  in  1978.  My  dissertation,  funded  by 
the  EPA,  analyzed  the  recreation  benefits  associated  with  the  abatement  of  water  pollution  in 
Boston  Harbor. 

Since  joining  the  faculty  at  Berkeley  in  1976. 1  have  taught  and  conducted  research  in  the 
fields  of  environmental  economics  and  resource  economics,  with  special  emphasis  on  cost-benefit 
analysis  and  what  is  known  as  non-market  valuation.  I  have  published  articles  and  reports  in  this 
field  and  I  have  conducted  many  academic  and  consulting  studies. 

Since  1988,  I  have  been  a  consultant  to  the  California  Attorney  General's  Office  and 
NOAA  on  various  natural  resource  damage  cases  in  California.  I  served  as  a  consultant  to  the 
Alaska  Department  of  Law  on  its  suit  for  natural  resource  damages  following  the  Exxon  Valdez 
oil  spill.  I  have  conducted  studies  on  the  economic  value  of  sportfishing  for  the  Alaska 
Department  of  Fish  &  Game,  on  the  economic  value  of  wetland  preservation  for  the  Inter- Agency 
San  Joaquin  Valley  Drainage  Program,  and  on  the  public  trust  values  associated  with  Mono  Lake 
for  the  California  State  Water  Resource  Control  Board's  EIR  in  connection  with  its  recent  Mono 
Lake  Water  Rights  Decision.  Earlier,  I  served  as  the  Board's  principal  economic  advisor  for  the 
first  three  years  of  its  Hearings  on  Water  Diversions  from  the  San  Francisco  Bay/Delta. 

Cases  in  which  I  have  testified  or  been  deposed  are  listed  in  an  Appendix. 

My  resume  is  attached  as  another  Appendix. 


2.   Overview  of  this  Report 

I  was  asked  to  review  a  report  entitled  "Volume  III:  Report  on  Potential  Economic  Losses 
Associated  with  Recreation  Services  in  the  Upper  Clark  Fork  River  Basin"  by  William  H. 
Desvousges  and  Steven  M.  Waters  of  Triangle  Economic  Research,  Durham  NC,  dated  July  13, 
1995  ("the  TER  Report").  This  is  a  study  of  the  economic  loss  to  anglers  in  Montana  resulting 
from  hazardous  releases  in  Upper  Clark  Fork  River  Basin.  The  TER  Report  was  accompanied 
by  Appendices  A  -  D,  plus  several  related  documents,  as  follows: 

Volume  II:  Critique  of  the  State  of  Montana's  Recreation  Study.  Expert  Report  of  WiUiam 
H.  Desvousges  and  Steven  M.  Waters,  July  13,  1995. 

Expert  Report  of  Daniel  L.  McFadden,  July  12,  1995. 

Research  Triangle  Institute,  Compensable  Value  Damages:  ARCO  Response  to  State  of 
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Montana  Damage  Assessment  Report  Upper  Clark  Fork  River  Basin.  Volume  II  ARCO's 
Compensable  Value  Damage  Estimates.  January  1994 

Research  Triangle  Institute,  Compensable  Value  Damages:  ARCO  Response  to  State  of 
Montana  Damage  Assessment  Report  Upper  Clark  Fork  River  Basin.  Appendix  A:  Random 
Utility  Models.  Appendix  B:  Telephone  Survey,  January  1994 

Research  Triangle  Institute,  Compensable  Value  Damages:  ARCO  Response  to  State  of 
Montana  Damage  Assessment  Report  Upper  Clark  Fork  River  Basin.  Appendix  D:  Mail  Survey 
Results,  January  1994 

RCG/Hagler  Bailly,  Assessment  of  Damages  to  Anglers  and  Other  Recreators  from 
Injuries  to  the  Upper  Clark  Fork  River  Basin,  January  1995 

RCG/Hagler  Bailly,  Compensable  Natural  Resource  Damage  Determination,  Upper  Clark 
Fork  River  NPL  Sites,  January  1995. 

Deposition  of  Janet  Decker-Hess,  Volumes  I  &  11,  March  21  &  22,  1995. 

In  addition  to  these  documents,  I  was  given  floppy  disks  with  various  computer  files  from 
Triangle  Economic  Research  (TER),  as  well  as  various  internal  memoranda  and  papers  by  TER 
staff  and  consultants  that  had  been  obtained  through  the  discovery  process.  I  subsequently 
obtained  several  reports  from  the  Montana  Department  of  Fish,  Wildlife  and  Parks  (DFWP) 
which  will  be  mentioned  below. 

I  have  not  seen  the  complete  set  of  data  collected  by  TER  in  the  course  of  preparing  its 
analysis  of  economic  loss  to  recreation  in  the  Upper  Clark  Fork  River  Basin.  What  I  have  seen 
represents  data  that  were  used  in  the  final  analysis  for  the  TER  Report  plus  a  portion  of  the 
earlier  data.  Likewise,  I  have  seen  some,  but  by  no  means  all,  of  the  computer  programs  used 
by  TER  in  the  course  of  its  analysis.  The  observations  here  represent  my  preliminary  assessment 
based  on  the  information  that  I  have  received  so  far.  I  reserve  the  right  to  modify  this  report  in 
any  respect  as  I  receive  new  information. 

My  review  of  TER's  estimate  of  the  economic  loss  to  recreation  associated  with  natural 
resource  injuries  in  the  Upper  Clark  Fork  River  Basin  consists  of  four  components.  In  Section 
3,  I  review  the  surveys  that  provided  much  of  the  data  for  the  analysis  in  the  TER  Report.  In 
Section  4,  I  review  the  variables  used  in  TER's  statistical  model  of  sport  fishing.  In  Section  5, 
I  review  the  structure  and  estimation  of  the  model.  In  Section  6,  I  review  the  interpretation  of 
the  model  results  and  their  extrapolation  to  the  estimation  of  aggregate  recreation  losses  in  the 
Upper  Clark  Fork  River  Basin. 

In  forming  my  opinions,  I  have  been  assisted  by  my  graduate  student  Craig  Mohn  and  his 
wife,  Mariko  Tanaka.  Craig  carried  out  computer  analyses  under  my  direction.  Mariko  reviewed 
maps  and  books  and  compiled  a  map  of  the  TER  sites  under  my  direction.  I  wrote  this  report 


and  it  represents  my  own  opinions. 

In  addition  to  the  reports  mentioned  above,  and  other  items  to  be  mentioned  below,  for 
general  information  about  fishing  in  Montana  I  consulted  the  following  books: 

Delorme  Mapping,  Montana  Atlas  &  Gazetteer,    1994. 

Hank  Fischer,  The  Floater's  Guide  to  Montana  Helena:  Falcon  Press  Publishing  Co,  1986. 

Chuck  Fothergill  and  Bob  Sterling,  The  Montana  Angling  Guide  Woody  Creek:  Stream 
Stalker  Publishing,  1988. 

John  Holt,  Montana  Fly  Fishing  Guide,  Volume  1:  West  of  the  Continental  Divide 
Helena:  Greycliff  Publishing  Co,  1995. 

Dick  Konizeski,  The  Montanans'  Fishing  Guide.  Volume  I:  Montana  Waters  West  of  the 
Continental  Divide   Missoula:  Mountain  Press  Publishing  Company,  4th  edition,  1992. 

Dick  Konizeski,  The  Montanans'  Fishing  Guide.  Volume  II:  Montana  Waters  East  of 
Continental  Divide  Revised  and  updated  by  James  A  Derleth.  Missoula:  Mountain  Press 
Publishing  Company,  4th  edition,  1992. 

Charles  Meek  and  Greg  Hoover,  Great  Rivers  -  Great  Hatches  Harrisburg:  Stackpole 
Books,  1992. 

W.  C.  McRae  and  Judy  Jewell,  Montana  Handbook  Chico:  Moon  Publications,  Second 
edition,  1994. 


3.    The  MOR  Survey 

3.1  Introduction 

In  any  empirical  research,  the  data  are  crucial.  Even  the  most  dazzling  theoretical  model 
will  come  to  naught  if  it  is  applied  to  unreliable,  unrepresentative  or  contaminated  data  that 
cannot  be  cleansed  of  these  defects.  It  is  appropriate,  therefore,  to  start  my  review  of  TER's 
analysis  of  recreation  losses  by  considering  the  data  on  which  it  is  based.  These  data  come  from 
a  special  survey  conducted  by  the  Research  Triangle  Institute  (RTI)  which  it  named  the  Montana 
Outdoor  Recreation  (MOR)  Survey.  The  MOR  survey  was  conducted  in  two  phases.  The  first 
phase,  conducted  in  July  1992,  was  a  screening  interview  in  which  a  random  sample  of 
individuals  in  the  state  of  Montana  were  contacted  by  telephone  and  asked  whether  anybody  in 
their  household  aged  16  or  over  participated  in  outdoor  recreation.  If  some  members  of  the 
household  did  participate  in  recreation,  the  interviewer  selected  one  of  them  to  serve  as  the 
respondent  for  the  survey.  The  interviewer  asked  the  respondent  first  for  his/her  opinions  about 
outdoor  recreation  in  the  state  of  Montana,  and  then  whether  he/she  had  participated  in  7  specific 
outdoor  recreation  activities  during  the  past  12  months,  and  how  much.  Next,  respondents  were 
asked  whether  they  planned  to  participate  in  any  of  those  activities  over  the  next  12  months,  and 
whether  they  planned  to  live  in  Montana  during  that  time.  If  they  answered  "yes"  to  both 
questions,  they  were  asked  if  they  would  volunteer  to  join  a  panel  of  people  who  would  be  paid 
$35  to  record  brief  summaries  of  all  their  visits  to  recreation  sites  over  the  next  year.  The 
interview  concluded  with  a  series  of  questions  about  demographic  and  economic  variables. 

The  second,  panel  phase  of  the  MOR  Survey  was  for  those  respondents  to  the  telephone 
survey  who  had  agreed  to  participate  in  the  recreation  panel.  Seven  times  over  the  next  14 
months  they  were  mailed  a  $5  check  together  with  a  trip  summary  booklet  covering  a  two-month 
period  (wave),  starting  with  July/August  1992  and  continuing  through  July/ August  1993.  They 
were  asked  to  record  in  the  booklet  information  on  all  recreation  trips  during  the  two-month 
period.  For  this  purpose,  the  booklet  was  divided  into  9  sections:  fishing,  boating  and  water 
sports,  camping,  hiking  and  trail  activities,  game  hunting,  waterfowl  hunting,  recreational 
trapping,  other  activities,  and  multiple  destination  trips.  A  large  map  was  included  along  with 
each  booklet,  and  respondents  were  requested  to  place  a  numbered  sticker  on  the  map  to  mark 
the  precise  destination  of  each  trip.  The  booklets  were  to  be  returned  in  a  pre-paid  envelope  at 
the  end  of  the  two-month  period.  Panel  members  who  did  not  return  their  booklets  on  time  were 
sent  reminder  postcards,  which  included  a  toll-free  phone  number  to  call  with  any  questions.  Four 
to  six  weeks  following  the  postcard  mailing,  follow-up  telephone  calls  were  made  to  all  panel 
members  who  still  had  not  returned  books.  A  comprehensive  data  management  system  was 
established  at  RTI  to  track  the  progress  of  the  survey  and  ensure  quality  control,  covering  mailing 
of  booklets,  receiving  of  booklets,  editing  of  booklets,  and  keying  of  data. 

Taken  together,  the  two  phases  of  the  MOR  Survey  constitute  a  very  substantial  survey 
effort.  I  have  been  conducting  recreation  and  other  surveys  since  1974,  and  I  am  not  aware  of 
any  private  recreation  survey  that  was  so  extensive  or  elaborate.  Having  a  survey  organization 
like  RTI  conduct  a  7-wave  panel  for  a  statewide  population   is  an  extremely  expensive 


undertaking.  If  one  can  afford  the  cost,  and  if  it  is  done  correctly,  a  panel  survey  offers  important 
advantages.  Using  a  random  sample  of  all  the  households  in  the  area  of  interest  is  a  more 
straightforward  approach  to  sampling  than  conducting  an  intercept  survey  of  recreationists  on-site. 
It  is  much  more  costly  since  one  needs  a  substantially  larger  sample  to  get  the  same  information 
about  recreation  activities  but,  in  principle,  it  can  avoid  a  variety  of  problems  and  confounding 
factors  that  otherwise  need  to  be  adjusted  for,  such  as  the  over-sampling  of  avid  recreationists. 
Likewise,  the  panel  approach  in  which  respondents  keep  a  "diary"  of  their  activities  provides 
more  complete  and  accurate  information;  as  long  as  they  continue  to  participate  in  the  survey, 
it  allows  one  to  track  respondents'  behavior  over  the  entire  season  or  year. 

However,  it  is  important  to  recognize  that  the  outcomes  of  surveys  can  sometimes  differ 
from  these  ideals.  Sampling  people  is  inevitably  far  more  complicated  than  sampling  inanimate 
objects.  People  may  not  be  willing  to  participate  in  a  survey,  they  may  wander  off,  or  they  may 
lose  interest  half  way  through.  And,  if  this  happens,  it  may  not  happen  at  random.  The  people 
who  do  this  may  be  systematically  different  from  those  who  don't,  but  not  necessarily  in  ways 
that  can  readily  be  identified  and  measured  by  the  analyst.  Real-world  sampling  sometimes 
produces  outcomes  that  are  not  what  one  expects.  Therefore,  when  one  assesses  the  MOR  Survey, 
it  is  important  to  look  at  not  only  the  amount  of  effort  that  went  into  the  survey  but  also  the 
nature  and  quality  of  the  output  -  how  well  did  the  survey  perform  in  practice,  and  what  did  it 
produce?  To  answer  these  questions,  I  did  two  things:  I  examined  the  data  from  the  MOR 
Survey,  and  I  checked  it  against  information  that  is  available  from  other  sources. 

In  addition  to  the  MOR  Survey  and  the  angler  survey  conducted  by  Hagler  Bailly  (1995) 
for  the  State  of  Montana,  there  are  at  least  two  other  major  sources  of  information  about  outdoor 
recreation  in  Montana  -  the  1991  National  Survey  of  Fishing,  Hunting,  and  Wildlife-Associated 
Recreation  (FHWAR),  conducted  by  the  U.S.  Bureau  of  the  Census  for  the  U.S.  Department  of 
Interior's  Fish  and  Wildlife  Service  (U.S.  Department  of  the  Interior,  1993);  and  statewide  mail 
surveys  of  angling  pressure  conducted  by  the  Montana  Department  of  Fish,  Wildlife  and  Parks 
in  1989,  1991  and  1993.  Brief  descriptions  of  them  follow. 

The  1991  National  Survey  of  FHWAR  was  the  eighth  such  survey  conducted  since  1955 
with  the  purpose  of  gathering  information  about  the  activities  of  hunters,  anglers  and 
nonconsumptive  participants;  the  information  is  regarded  as  an  important  tool  by  natural  resource 
managers  and  industry  planners.  The  1991  survey  was  conducted  in  two  phases.  The  first, 
screening  phase,  conducted  in  January  and  February  1991,  interviewed  a  sample  of  129,500 
households  nationwide,  primarily  by  telephone,  to  determine  who  in  the  household  had  fished, 
hunted  or  engaged  in  nonconsumptive,  wildlife-related  activity  during  1990,  and  who  planned  to 
engage  in  those  activities  in  1991.  In  Montana  in  this  phase,  about  1,070  completed  household 
interviews  were  obtained,  for  a  response  rate  of  approximately  93.6%.  Roughly  68%  of  the 
interviewed  households  were  contacted  by  telephone;  the  remainder  were  contacted  by  personal 
visit.  The  second,  more  in-depth  phase  covered  recreation  during  1991.  This  phase  involved  three 
detailed  interviews  conducted  every  four  months  with  samples  of  likely  anglers,  hunters  and 
nonconsumptive  participants  who  were  identified  in  the  initial  screening.  These  interviews  were 
conducted  mainly  by  telephone,  with  in-person  interviews  for  those  respondents  who  could  not 


be  reached  by  telephone.  Respondents  in  this  phase  were  limited  to  those  at  least  16  years  old. 
Each  respondent  provided  information  pertaining  only  to  his/her  activities.  Sample  sizes  were 
designed  to  provide  statistically  reliable  results  at  the  State  level  for  fishing,  hunting,  and 
nonconsumptive  activities.  In  Montana,  about  1220  persons  were  designated  for  interviews  in  the 
second  phase  of  the  survey  and  about  1 180  of  these  completed  interviews,  for  a  response  rate  of 
96.9%.  It  is  the  information  from  these  interviews  in  the  second  phase  that  is  reported  in  U.S. 
Department  of  the  Interior  (1993). 

The  Montana  Department  of  Fish,  Wildlife  and  Parks  has  conducted  statewide  angling 
mail  surveys  from  time  to  time  in  the  past.  In  1989,  the  Montana  Legislature  approved  funding 
for  an  "Enhanced  Survey  of  Angling  Pressure"  to  be  conducted  every  other  year.  Since  then, 
surveys  have  been  conducted  in  1989,  1991,  and  1993  using  essentially  the  same  format.  The 
results  of  the  1993  survey  are  still  in  press.  For  my  analysis,  therefore,  I  focus  on  the  1991 
survey  (McFarland  and  Hughes  1994).  The  survey  involved  mailing  questionnaires  to  a  stratified 
random  sample  of  resident  and  nonresident  anglers  each  month  covering  the  12-month  period 
from  March  1991  to  February  1992.  Respondents  were  asked  whether  they  had  fished  in  Montana 
during  the  previous  month  and,  if  so,  they  were  requested  to  list  each  water  that  they  had  fished, 
the  date,  the  number  of  days  of  fishing,  and  the  mode  of  fishing.  The  samples  were  drawn  from 
holders  of  resident  and  nonresident  fishing  licenses.  For  the  residents  and  some  of  the 
nonresidents,  fresh  samples  were  drawn  monthly  as  records  of  fishing  license  sales  arrived  in 
Helena.  Nonresidents  with  2-day  licenses  were  sampled  once,  in  February,  about  their  fishing  for 
the  entire  license  year.  Over  the  12  months  of  the  survey,  questionnaires  were  mailed  to  79,120 
residents  and  16,230  non-residents,  of  which  71,017  were  successfully  delivered  to  residents  and 
14,205  to  nonresidents.  Responses  were  received  from  45,384  residents  and  8,764  nonresidents, 
representing  response  rates  of  63.9%  and  61.7%,  respectively. 

The  remainder  of  this  section  of  my  report  is  organized  as  follows.  In  Section  3.2  I  review 
the  first  phase  of  the  MOR  survey,  the  screener  survey  of  a  random  sample  of  Montana 
residential  telephone  numbers.  In  Section  3.3  I  discuss  the  recruitment  of  people  identified  as 
ehgible  in  the  telephone  survey  to  participate  in  the  14-month  panel  survey.  In  Section  3.4  I 
review  the  panel  phase  of  the  MOR  survey.  My  conclusions  are  summarized  in  Section  3.5. 


3.2  Telephone  Phase  of  the  MOR  Survey 

In  its  general  structure,  RTFs  MOR  survey  is  rather  similar  to  the  Census  Bureau's 
National  Survey  of  FHWAR:  both  involved  a  panel  of  individuals  who  reponed  on  outdoor 
recreation  in  Montana,  in  one  case  during  calendar  1991  and,  in  the  other,  during  the  period  July 
1992  -  August  1993.  In  both  cases,  the  panel  was  recruited  through  a  preliminary,  screener  survey 
of  the  general  population.  The  MOR  survey  involved  telephone  interviews  with  a  sample  of  2,071 
households,  obtained  via  random  digit  dialing,  representing  a  response  rate  of  73%;  the  FHWAR 
survey  involved  telephone  or  in-person  interviews  with  a  random  sample  of  1,070  households, 
selected  from  samples  used  for  the  Census  Bureau's  Current  Population  Survey,  representing  a 
response  rate  of  93.6%.  In  each  case,  the  chief  purpose  of  the  screener  survey  was  to  pave  the 


way  for  the  subsequent  panel  survey  by  identifying  the  target  population  of  interest,  which  was 
approximately  the  same  in  both  surveys  ~  the  population  of  Montana  residents  aged  16  and  older 
who  planned  to  engage  in  outdoor  recreation  during  the  period  of  the  panel  survey.  These  were 
mainly  people  who  had  participated  in  those  same  activities  —  in  the  sense  of  having  taken  at 
least  one  such  outdoor  recreation  trip  —  during  the  period  covered  by  the  screener  interview, 
namely  calendar  1991  for  the  Census  Bureau  and  July  1991  -  June  1992  for  the  MOR  survey. 
Hence,  the  screener  survey  is  a  guarantor  of  the  integrity  of  the  panel  survey.  If  it  generates  a 
sample  that  is  fully  representative  of  the  target  population,  this  augurs  well  for  the  accuracy  and 
reliability  of  the  panel  survey.  The  converse  is  true  if  it  generates  a  sample  that  poorly  represents 
the  target  population. 

As  indicated,  the  MOR  screener  survey  employed  a  larger  sample  than  the  FHWAR 
screener  survey,  but  it  also  had  a  lower  response  rate.  Another  difference  was  survey  mode  —  the 
MOR  screener  was  conducted  entirely  by  telephone,  while  the  Census  Bureau  used  in-person 
interviews  for  almost  one  third  of  its  sample.  Professor  McFadden  has  mentioned  two  features 
of  the  MOR  telephone  survey  that  cause  him  some  minor  concern.  The  first  is  that  telephone 
surveys  may  not  have  provided  a  fully  representative  sample.  By  definition,  households  without 
telephones  were  not  included  in  the  MOR  sample,  and  households  with  multiple  telephone  lines 
were  more  likely  to  be  sampled  than  those  with  but  one  line.  Similarly,  households  who  are  not 
at  home  were  much  less  likely  to  be  contacted  and  included  in  the  MOR  sample  than  households 
who  stay  at  home  more.  Secondly,  Professor  McFadden  notes  that  the  screening  of  household 
members  for  the  panel  could  have  introduced  a  degree  of  arbitrariness  in  the  MOR  sample.  The 
probability  of  being  selected  depended  on  the  number  of  adult  recreators  in  a  household.  And  the 
adult  recreators  within  the  household  who  were  at  home  at  the  time  of  the  screening  call  tended 
to  be  included  more  readily  than  those  who  were  not.  Considerations  such  as  these  may  have  led 
the  Census  Bureau  to  use  in-person  interviews  for  the  sake  of  a  more  representative  sample  in 
its  FHWAR  survey. 

Professor  McFadden  states  that  the  sampling  procedure  employed  by  RTI  was  sufficiently 
straightforward  that  appropriate  weights  can  be,  and  were,  developed  such  that  the  weighted 
sample  in  the  MOR  survey  is  representative  of  the  target  population.  To  see  whether  this  is  so, 
one  should  look  at  the  data.  The  TER  Report  uses  the  weights  to  extrapolate  from  the  MOR 
telephone  sample  of  2071  respondents  to  the  entire  adult  population  of  Montana.  In  Table  A- 10, 
it  provides  estimates  of  the  number  of  residents  of  Montana  aged  16  and  over  who  took  one  or 
more  outdoor  trips  for  each  of  the  seven  recreation  activities  covered  by  the  survey  during  the 
12  months  prior  to  the  telephone  survey.  These  would  constitute  the  target  population  for  the 
subsequent  MOR  panel  survey.  Thus,  for  example,  the  TER  Report  estimates  that  326,046  adult 
Montana  residents  participated  in  fishing  in  1991/92.  In  producing  these  estimates,  the  TER 
Report  assumed  a  total  adult  population  in  Montana  of  583,914  (Table  A- 13).  In  fact,  according 
to  the  Census  Bureau,  in  1991  there  were  about  600,900  persons  aged  16  and  over  in  the  state 
of  Montana.  The  difference  is  small  --  about  2.9%  -  but  it  implies  that  the  TER  estimates  of 
participation  should  be  increased  by  something  like  that  proportion  to  benchmark  them  against 
the  actual  1991  population.  This  presumably  would  push  up  the  TER  estimate  of  adult  anglers 
residing  in  Montana  in  1991  from  326,046  to  something  like  335,500. 


As  I  noted  earlier,  there  are  two  other  sources  of  information  about  participation  in 
outdoor  recreation  by  Montana  residents  in  1991,  the  Census  Bureau's  National  Survey  of 
FHWAR  and  the  State  of  Montana  Department  of  Fish,  Wildlife  and  Parks  (DFWP)  statewide 
mail  survey  of  angling  pressure.  The  FHWAR  survey  covers  fishing,  hunting,  and  non- 
consumptive  wildhfe-associated  recreation.'  The  Montana  DFWP  survey  covers  just  fishing.  For 
fishing,  the  Census  Bureau's  estimate  of  the  number  of  adult  anglers  in  the  state  of  Montana  in 
1991  is  170,800,  with  a  90%  confidence  interval  of  about  154,300  to  187,300  (see  Table  1).  Even 
if  one  takes  the  upper  end  of  range,  this  is  substantially  less  than  TER's  estimate  of  the  same 
target  population.  Relative  to  the  Census  Bureau's  finding,  TER  overestimates  the  number  of 
adult  anglers  in  Montana  in  1991  by  74%  if  one  does  not  adjust  for  its  underestimate  of  the  state 
population,  and  by  about  79%  if  one  does.  The  Montana  DFAVP  also  overestimates  the  number 
of  adult  anglers  in  Montana  in  1991,  but  to  a  much  smaller  degree.  Its  estimate  of  218,567  adult 
resident  anglers  is  17%  larger  than  the  upper  end  of  the  Census  Bureau's  range. 

On  the  basis  of  these  data,  I  would  be  inclined  to  put  the  number  of  adult  resident  anglers 
in  Montana  in  1991  at  around  the  upper  end  of  the  Census  Bureau's  range,  namely  about 
187,000. 1  am  not  necessarily  surprised  that  the  Montana  DFWP  survey  produces  a  slightly  larger 
total,  because  it  is  based  on  counts  of  license  holders.  There  will  always  be  some  individuals  who 
purchase  fishing  licenses  because  they  hope  or  intend  to  go  fishing  during  the  coming  season  but 
then  other  things  intervene  ~  they  are  too  busy,  they  become  sick,  they  move  away,  etc  —  and 
they  never  do  get  to  go  fishing.  Therefore,  I  would  expect  the  number  of  license  holders  to 
exceed  the  number  of  actual  anglers  (defined  as  people  who  make  at  least  one  fishing  trip  during 
the  year)  by  at  least  some  percentage.  However,  this  does  not  apply  to  surveys  such  as  the  MOR 
survey  which  are  based  on  the  general  population  rather  than  a  sample  frame  of  license  holders. 
The  substantial  error  in  TER's  estimate  of  anglers  in  Montana  is,  therefore,  all  the  more 
surprising. 

Hunting  is  another  activity  where  one  can  independently  verify  the  TER  Report's  estimate 
of  a  target  population  for  the  MOR  panel  survey.  The  FHWAR  survey  provides  an  estimate  of 
the  number  of  adult  residents  who  in  1991  participated  in  some  form  of  hunting  and,  separately, 
in  big  game  hunting,  small  game  hunting,  migratory  bird  hunting,  and  hunting  for  other  animals. 
The  TER  Report  provides  separate  weighted  population  estimates  for  waterfowl  hunting  and  game 
hunting.  From  the  definitions  provided,  the  Census  Bureau's  "migratory  bird"  category  appears 
to  coincide  with  RTFs  "waterfowl"  category.  Therefore,  these  two  estimates  of  participation 
should  be  comparable.  As  shown  in  Table  1,  TER  estimates  that  about  114,900  adult  residents 
of  Montana  participated  in  waterfowl  hunting  in  1991/92.  The  Census  Bureau's  estimate  is  13,500 
(it  gives  no  standard  error  or  confidence  interval  in  this  case).  The  difference  is  quite  striking. 
Relative  to  the  Census  Bureau's  finding,  TER  overestimates  the  number  of  adult  waterfowl 
hunters  in  Montana  in  1991  by  about  751%  if  one  does  not  adjust  for  its  underestimate  of  the 


The  last  is  defined  as  observing,  photographing  or  feeding  fish  or  other  wildlife,  excluding  trips  to  zoos, 
circuses,  aquariums,  and  museums. 


TABLE  1 :  NUMBER  (in  ttx)usands)  OF  MOMTANA  RESIDENTS  AGED  1 6  AND  OLDER  PARTICIPATING  IN  FISHING  &  HUNTING 


ACTIVITY 

FISHING 
ALL  HUNTING 
GAME  HUNTING 
WATERFOWL  HUNTING 


1992  RTI  PHONE  SURVEY 

1991  CENSUS  BUREAU 

1991  STATEWIDE  ANGLING 

FHWAR  SURVEY 

PRESSURE  SURVEY 

(1) 

(2) 

(3) 

326.0 

170.8     +/-16.5 

218.6 

254.2" 

157.6    +/-14.8 

246.5 

114.9 

13.5 

SOURCE:        (1)  TER  (1995),  Table  A-10 

(2)  1991  National  Survey  of  FHWAR:  Montana,  Tables  2,  D-1,  D-2 

(3)  McFarland  (1994a),  Table  12 
*•  Estimated  from  MOR  data 


state  population,  and  by  about  775%  if  one  does. 

Since  the  MOR  telephone  survey  combines  small  and  big  game  hunting  into  a  single 
category,  whereas  the  FHWAR  survey  keeps  them  separate,  one  cannot  directly  compare  their 
findings.  However,  it  is  evident  from  Table  1  that,  relative  to  the  Census  Bureau,  the  TER  Report 
overestimates  participation  in  game  hunting.  The  TER  Report  estimates  that  about  246,500  adult 
residents  of  Montana  participated  in  game  hunting  in  1991/92,  while  the  Census  Bureau  estimates 
that  only  about  157,600  adult  residents  participated  in  any  form  of  hunting  in  1991,  with  a  90% 
confidence  interval  from  142,400  to  172,400.  To  obtain  a  more  direct  comparison  between  the 
surveys,  I  aggregated  the  MOR  telephone  survey's  two  categories  of  game  hunting  and  waterfowl 
hunting.  If  anything,  this  should  produce  an  underestimate  of  the  Census  Bureau's  "hunting" 
category  since  it  may  omit  some  types  of  hunting  covered  by  the  Census  Bureau.  In  the  MOR 
survey,  908  of  2071  respondents  reported  having  participated  at  least  once  in  game  hunting 
and/or  waterfowl  hunting  during  the  past  12  months.  At  this  time,  I  do  not  have  the  specific 
weights  that  TER  used  to  extrapolate  from  the  individuals  in  the  MOR  telephone  survey  to  the 
entire  adult  population  of  Montana;  these  apparently  were  not  part  of  the  data  that  were  provided 
to  me.  Based  on  what  I  have  been  able  to  deduce  about  TER's  weights,  I  would  estimate  that  the 
908  respondents  in  the  phone  survey  would  extrapolate  to  about  254,240  hunters  in  the  entire 
adult  population  of  Montana.  Relative  to  the  upper  bound  of  the  Census  Bureau's  confidence 
interval  for  resident  adult  hunters,  this  would  be  an  overestimate  of  about  47%. 

In  addition  to  participation,  the  MOR  telephone  survey  generated  information  covering 
a  variety  of  attitudinal,  demographic,  and  economic  variables  pertaining  to  adult  outdoor 
recreators  in  Montana  ~  their  age,  sex,  income,  education,  whether  they  owned  a  boat,  a  cabin 
or  a  hunting  lodge,  what  type  of  vehicle  they  used  for  recreation,  etc.  Age,  income,  and  sex  also 
happen  to  be  variables  reported  in  the  National  Survey  on  FHWAR.  Unfortunately,  the  ranges 
that  RTI  used  in  the  MOR  telephone  survey  for  recording  age  and  income  are  different  from 
those  employed  by  the  Census  Bureau.  However,  a  direct  comparison  is  readily  available  for  sex. 
The  results  are  shown  in  Table  2.  While  males  account  for  49%  of  the  population  aged  16  and 
older  in  Montana,  they  account  for  a  larger  percentage  of  adult  anglers  and  hunters  because  those 
activities  are  much  more  favored  by  males  than  females  in  that  state.  The  Census  Bureau  reports 
that  males  account  for  75%  of  the  anglers  and  87%  of  the  hunters  in  Montana.  By  contrast,  when 
extrapolated  to  the  adult  population  of  Montana,  the  MOR  telephone  survey  implies  that  males 
account  for  61%  of  the  anglers  and  70%  of  the  hunters  in  the  state.  This  is  clearly  inconsistent 
with  the  Census  Bureau's  findings. 

In  summary,  there  is  a  systematic  pattern  whereby  the  MOR  telephone  survey 
overestimates  the  size  of  the  target  populations  of  outdoor  recreationists  in  Montana  in  1991/92, 
sometimes  to  an  enormous  degree,  and  overstates  the  extent  to  which  they  are  composed  of 
females.  In  the  light  of  this  evidence,  I  conclude  that  the  MOR  telephone  survey  failed  to  meet 
its  goal  of  providing  a  representative  sample  of  the  target  population  of  outdoor  recreationists  in 
Montana.  The  magnitude  of  this  failure  is  quite  striking. 

In  the  remainder  of  this  section  I  discuss  some  possible  reasons  for  the  failure,  and  then 


TABLE  2:  SEX  OF  M0NTA^4A  RESIDENT  ANGLERS  AND  HUNTERS,  1 991 
1 991  CENSUS  BUREAU  1 992  RTI 


FHWAR  SURVEY 

MOR  SURVEY 

AM(^I  FRC: 

(1) 

(2) 

%  Male 

75.1% 

61.0% 

%  Female 

24.9% 

39.0% 

HUN  1 ERS 

%  Male 

86.6% 

70.0% 

%  Female 

13.4% 

30.0% 

SOURCE:  (1)  1991  National  Survey  of  FHWAR:  Montana,  Table  13 
(2)  Estimated  from  MOR  data 


comment  on  what  I  see  as  its  implications. 

Tiiere  are  at  least  two  factors  that  could  have  caused  the  errors  in  the  MOR  telephone 
survey.  One  is  the  phenomenon  known  in  the  survey  research  literature  as  telescoping,  which  is 
the  tendency  of  respondents  when  recalling  past  events  to  relocate  them  to  either  an  earlier  or 
later  period  than  when  they  actually  occurred  (Neter  and  Waksberg,  1964).  The  latter  has  often 
been  found  the  more  serious  problem  —  people  report  as  happening  within  the  last  year,  say, 
things  that  had  actually  happened  earlier,  leading  to  an  exaggerated  estimate  of  their  frequency 
(Loftus  et  al.,  1990).  Thus,  respondents  who  had  gone  fishing  in  the  past,  but  not  within  the 
previous  12  months,  might  have  erroneously  reported  that  they  had  fished  in  the  MOR  telephone 
survey.  Another  explanation  (these  are  not  mutually  exclusive)  is  that  the  survey  might  have 
mistakenly  identified  as  anglers  or  hunters  individuals  who  were  neither.  This  is  consistent  with 
both  the  exaggerated  numbers  of  anglers  and  hunters  and  also  the  exaggerated  proportion  of 
females.  In  effect,  there  are  two  populations:  the  population  of  anglers,  hunters,  etc,  which  is 
highly  skewed  to  males,  and  the  population  of  non-anglers  and  non-hunters,  which  in  Montana 
is  about  38%  males  and  62%  females.  The  purpose  of  the  telephone  survey  is  to  identify  the  first 
population  but,  by  mistake,  it  also  picks  up  some  members  of  the  second.  This  has  two 
consequences:  it  produces  an  overestimate  of  the  size  of  the  target  population,  and  one  which  is 
less  heavily  weighted  to  males.  This  can  be  thought  of  as  a  contamination  phenomenon:  the  target 
population  is  contaminated  with  some  members  of  a  non-target  population.  I  estimate  that  this 
contamination  could  account  for  an  upward  bias  of  about  106,000  (56%)  in  TER's  estimate  of 
adult  resident  anglers  in  Montana. 

What  could  have  brought  about  the  contamination?  It  may  have  arisen  as  a  result  of  errors 
in  the  design  of  the  MOR  telephone  survey  or  carelessness  in  its  execution.  For  example,  wives 
may  have  been  reporting  on  their  husband's  fishing,  and  this  was  erroneously  recorded  as  though 
they  were  reporting  on  their  own  fishing.  Or,  wives  who  accompanied  their  husband  on  a  fishing 
trip  but  didn't  themselves  fish  might  have  said  "yes"  when  the  interviewer  asked  if  they  had 
participated  in  fishing.  These  are  errors  that  can  be  caught  with  careful  wording  of  the 
questionnaire  and  vigilance  on  the  part  of  the  interviewer.^  It  is  possible  that  this  was 
compounded  by  the  exclusive  use  of  telephone  for  the  MOR  survey.  When  the  interviewer  is 
present  in  person,  as  in  a  third  of  the  Census  Bureau's  interviews,  it  is  easier  to  monitor  whether 
respondents  are  answering  the  question  as  intended,  and  to  probe  when  they  give  incomplete  or 
ambiguous  responses. 

Whatever  the  factors  that  caused  it,  I  believe  that  the  failure  of  the  MOR  telephone  survey 
has  serious  implications.  These  concern:  (1)  the  quality  of  the  analysis  that  went  into  the  TER 
Report;  (2)  the  reliability  of  the  weighting  scheme  employed  by  TER;  and  (3)  the  design  of  the 


^  The  TER  Report  conveys  the  impression  that  the  MOR  survey  was  initiated  with  some  haste.  It  mentions  that 
there  was  a  single  training  session  for  interviewers  on  the  day  before  the  telephone  survey  began,  which  consisted 
of  90  minutes  of  formal  training  followed  by  several  hours  of  hands-on  practice  with  the  instrument.  This  does  not 
appear  to  have  left  time  to  revise  the  interview  script  to  incorporate  feedback  from  the  interviewer  training. 
Inadequate  interviewer  training  could  certainly  help  explain  the  overcounting  of  female  anglers  and  hunters. 
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subsequent  MOR  panel  survey. 

With  regard  to  the  first,  it  is  both  common  sense  that,  when  one  measures  something  for 
which  other  researchers  have  also  obtained  measures,  one  checks  one's  own  results  against  theirs 
and  investigates  any  discrepancies.  Not  to  have  done  this  is  an  indictment  of  the  quality  of  the 
research.  In  the  case  of  the  TER  Report,  the  failure  to  reference  the  1991  National  Survey  of 
FHWAR  cannot  have  been  due  to  lack  of  information.  I  know  that  members  of  the  TER  research 
team  and  their  colleagues  attended  Annual  Meetings  of  the  W133  Environmental  Economics 
Group  at  which  representatives  of  the  Fish  and  Wildlife  Service  made  presentations  about  the 
1991  National  Survey  and,  on  at  least  one  occasion,  freely  distributed  copies  of  the  report.  If  not 
due  to  ignorance,  the  failure  of  TER  to  reference  the  National  Survey  must  be  attributed  to  either 
absent-mindedness  or  deliberate  design.  Either  way,  it  is  an  egregious  omission. 

With  respect  to  weighting,  while  there  is  general  agreement  about  the  desirability  of  using 
sampling  weights  when  the  aim  is  to  estimate  population-level  statistics,  two  qualifications  should 
be  noted.  First,  weighting  is  not  a  panacea  that  corrects  for  all  errors  in  survey  execution.  Second, 
the  right  weights  have  to  be  used.  The  MOR  telephone  survey  illustrates  the  problem.  The  MOR 
telephone  survey  data  were  weighted  to  adjust  for  differences  in  sampling  rates  by  region  and 
to  benchmark  against  known  values  for  various  population-level  demographic  variables,  such  as 
the  age  and  sex  distribution  of  adult  Montana  residents.  With  these  weights,  therefore,  the  2071 
respondents  blow  up  correctly  to  Montana  population  ~  for  example,  the  weighted  ratio  of  males 
to  females  for  the  full  sample  of  respondents  matches  the  true  ratio  for  adult  Montana  residents. 
But,  as  we  have  seen,  the  weighted  sample  fails  miserably  at  representing  the  population  of 
anglers  or  hunters  in  Montana.  If  the  aim  was  to  represent  outdoor  recreators  in  Montana  - 
which  clearly  was  the  target  population  for  recruiting  the  panel  in  the  next  phase  of  the  survey  - 
-  the  wrong  weights  were  being  used.  Too  many  people,  and  especially  too  many  women,  are 
wrongly  identified  as  being  anglers  or  hunters.  Whether  this  occurred  because  of  telescoping  or 
contamination,  the  weighting  is  inadequate  to  undo  errors  of  survey  execution.^ 

Finally,  as  I  show  below,  the  failure  of  the  MOR  telephone  survey  to  produce  an  accurate 
sample  of  the  target  population  of  outdoor  recreators  in  Montana  had  unforeseen  repercussions 
on  the  effectiveness  of  the  panel  survey  which  followed  it.  Because  of  the  failure  of  the  telephone 
survey,  the  TER  research  team  had  an  exaggerated  impression  of  the  number  of  anglers  and 
hunters  in  the  Montana  population.  Therefore,  they  overestimated  the  number  of  anglers  and 


^  This  is  not  an  isolated  occurrence.  Professors  Hausman  and  McFadden  ran  into  the  same  problem  in  their 
study  for  Exxon  of  the  impact  of  the  Exxon  Valdez  oil  spill  on  outdoor  recreation  by  residents  of  Alaska  (Hausman, 
Leonard  and  McFadden,  1993).  That  study  also  involved  a  two-phase  survey,  the  first  being  a  screening  survey  of 
a  random  sample  of  Alaskan  phone  numbers  to  identify  households  which  had  participated  in  various  types  of 
outdoor  recreation  in  Alaska  in  1988  and  1989,  and  the  second  being  a  follow-up  mail  survey  of  respondents 
identifying  themselves  as  recreational  participants.  On  the  basis  of  their  surveys.  Professors  Hausman  and  McFadden 
form  an  estimate  of  fishing  in  Alaska  by  residents  which  substantially  overstates  the  amount  of  fishing  relative  to 
the  estimates  of  the  National  Survey  of  FHWAR  and  other  sources.  Too  many  respondents  may  have  wrongly 
identified  themselves  as  having  participated  in  fishing  in  Alaska  in  1988  and  1989,  and  this  was  not  adequately 
corrected  by  the  weighting  scheme  that  Professors  Hausman  and  McFadden  employed. 
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hunters  in  the  panel  that  they  recruited  from  among  the  respondents  to  the  telephone  survey. 
Therefore,  their  panel  turned  out  to  be  less  informative  than  they  had  expected.  Given  the  true 
proportion  of  anglers  and  hunters,  their  panel  was  too  small  for  what  they  wanted  it  to 
accomplish. 


3.3  The  Recruitment  of  the  MOR  Panel 

At  the  end  of  the  MOR  telephone  survey,  respondents  were  asked  whether  they  planned 
to  participate  in  any  of  7  outdoor  recreation  activities  over  the  next  12  months,  and  whether  they 
planned  to  live  in  Montana  during  that  time.  If  they  answered  "yes"  to  both  questions,  they  were 
asked  if  they  would  volunteer  to  join  a  panel  of  people  who  would  be  paid  to  record  their 
recreation  activities  over  the  next  year.  Of  the  2071  respondents,  1529  were  eligible  to  join  the 
panel  because  they  met  both  criteria.'*  Of  these  eligibles,  1 134  reported  that  they  had  participated 
in  fishing  during  the  previous  year,  282  reported  that  they  had  not  participated  in  fishing,  and  113 
said  they  did  not  know  whether  they  had  participated  in  fishing.  I  will  treat  the  last  group  as  not 
having  participated.  Of  the  eligible  recreators,  1 149  agreed  to  participate  in  the  panel  survey;  881 
had  participated  in  fishing  during  the  previous  year,  while  268  had  not.  Thus,  anglers  were  more 
likely  (78%)  to  join  the  panel  than  non-fishing  recreators  (68%),  a  difference  which  is  statistically 
significant.  Hence,  anglers  are  over-represented  in  the  panel  relative  to  other  outdoor  recreators. 
This  is  a  flaw  in  a  survey  intended  as  a  representative  sample  of  outdoor  recreators  in  Montana. 


3.4  The  MOR  Panel  Survey 

Panel,  or  longitudinal,  surveys  are  recognized  by  survey  researchers  as  advantageous  for 
measuring  change  or  for  measuring  activities  that  occur  at  varying  rates  over  time  such  that 
measurement  at  a  single  point  in  time  would  produce  unreliable  or  unrepresentative  information. 
Although  dating  back  to  before  World  War  II,  the  use  of  panel  surveys  in  the  social  sciences 
greatly  expanded  in  the  late  1960's  and  early  1970's,  and  they  came  to  receive  considerable 
attention  from  professional  and  academic  researchers  (Duncan,  Juster  and  Morgan,  1986).  While 
panel  surveys  can  offer  considerable  advantages  in  providing  accurate  information  on  time- 
varying  phenomena,  there  are  also  some  potential  drawbacks.  One  is  cost  ~  the  expense  of 
creating  and  maintaining  a  panel  may  can  be  quite  substantial  and  may  lie  beyond  the  budget  of 
most  researchers.  Another  is  the  logistics  of  data  management  and  file  maintenance  for  large- 
scale  panels.  In  addition,  Sudman  and  Ferber  (1979)  mention  problems  of  getting  and  maintaining 
cooperation  and  data  accuracy: 

"The  first  problem  [getting  and  maintaining  cooperation]  is  one  that  can  seriously  affect 


^  The  TER  Report  says  that  1532  telephone  respondents  were  eligible  for  the  panel.  I  am  unable  to  replicate 
that  number  from  the  data  provided  to  me.  For  these  data,  I  find  that  1535  telephone  respondents  planned  both  to 
live  in  Montana  and  to  participate  in  outdoor  recreation  during  the  next  year.  However,  6  of  them  had  addresses  with 
zip  codes  that  would  make  them  nonresidents. 
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the  representativeness  of  a  panel  study.  If  panel  members  are  to  keep  written  records  ... 
the  initial  rate  of  cooperation  can  be  as  low  as  50%  even  by  personal  interview  and  may 
be  much  lower  if  cooperation  is  sought  by  mail.  Those  who  cooperate  are  more  likely  to 
be  more  educated,  in  clerical  or  professional  occupations,  in  the  middle  income  levels, 
and  in  the  younger  and  middle  age  brackets.  As  a  result,  a  panel  at  the  beginning  of  the 
operation  may  not  be  representative  of  the  population  from  which  it  was  selected.  This 
is  only  the  start  of  the  problem,  however.  Dropouts  can  be  substantial.  About  half  of  the 
people  that  even  consent  to  participate  in  a  panel  study  may  drop  out  after  the  first  two 
or  three  rounds,  especially  if  they  are  asked  to  keep  written  record.  As  a  result,  a  panel 
operation  can  become  increasingly  unrepresentative  of  the  population  from  which  it 
originally  came... 

The  accuracy  of  panel  member  reports  is  another  problem,  one  that  seems  to  vary  with 
the  type  of  study.  This  problem  is  most  serious  with  diary  studies,  where  panel  members 
requested  to  record  their  purchases  of  many  different  items  day  by  day  in  diaries  that  are 
turned  in  or  mailed  every  week. ..As  a  result,  the  accuracy  of  such  information  can  vary 
greatly  within  the  same  study  by  the  type  of  product  and  the  type  of  information  that  is 
being  sought." 

Another  potential  problem  in  panel  surveys  is  what  is  called  conditioning  or  time-in-sample  bias. 
This  is  mentioned  by  Sudman  and  Ferber  and  is  discussed  at  greater  length  by  Bailar  (1989),  who 
defines  it  as  "the  phenomenon  that  estimates  from  people  reporting  for  the  same  time  period  but 
with  different  levels  of  exposure  to  the  survey  have  different  expected  values. ...Bias  is  observed 
when  a  significantly  higher  or  lower  level  for  a  characteristic  occurs  on  the  first  interview  than 
in  subsequent  interviews,  when  one  would  expect  the  same  level."  This  occurs,  for  example,  in 
consumer  expenditure  diary  surveys  where  there  is  more  comprehensive  reporting  on  the  first  day 
of  the  week  than  later  in  the  week,  and  in  the  first  week  than  later  weeks.  Bailar  mentions  several 
factors  that  could  be  responsible  for  such  phenomena.  Respondents  might  become  trained  by  their 
exposure  to  the  survey  and  provide  more  accurate  data  in  later  interviews.  Respondents  might 
actually  change  their  behavior  during  the  course  of  the  survey.  Interviewers  might  not  provide 
the  same  stimulus  to  the  respondents  in  the  later  interviews  as  in  the  first  one.  Respondents  might 
learn  that  some  responses  mean  additional  questions,  so  that  start  avoiding  to  give  certain 
answers.  Respondents  might  become  bored  with  the  survey  process  and  stop  working  as  diligently 
or  energetically  at  providing  accurate  responses  in  later  interviews.  Whatever  the  precise  cause, 
Bailar  regards  time-in-sample  bias  as  a  major  concern  for  panel  surveys. 

The  existing  literature  on  panel  surveys  deals  mainly  with  items  other  than  outdoor 
recreation  but,  in  principle,  the  same  issues  could  apply  to  panels  of  recreationists.  It  is  important, 
therefore,  to  consider  how  effectively  the  MOR  survey  overcame  the  challenges  to  a  successful 
panel  survey.  It  certainly  appears  to  have  been  a  costly  and  well  funded  undertaking.  How  well 
did  it  overcome  the  other  challenges? 

My  ability  to  answer  this  question  is  limited  by  the  inadequacy  of  the  information 
provided  to  me.  The  computer  files  that  I  have  received  with  data  from  the  MOR  panel  survey 
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contain  mainly  data  used  for  estimating  the  fishing  demand  model  presented  in  the  TER  Report. 
The  model  uses  only  a  subset  of  the  data  collected  in  the  MOR  panel  survey,  and  the  files  in  my 
possession  constitute  only  a  portion  of  the  data  collected  in  the  MOR  panel  survey. 

In  addition  to  computer  files,  I  have  written  descriptions  and  analyses  of  data  from  the 
panel  survey  contained  in  the  TER  Report  and  in  other  documents  and  memos  generated  by  TER 
staff  and  consultants  that  were  obtained  by  the  State  of  Montana  and  provided  to  me.  These 
written  materials  provide  an  incomplete  account  of  the  panel  survey  and  its  findings.  For 
example,  the  description  of  the  results  of  the  MOR  panel  survey  in  the  TER  Report  is  confined 
to  the  fishing  portion  of  the  survey  and  —  for  reasons  that  are  never  explained  —  entirely  omits 
the  other  seven  outdoor  recreation  activities  that  were  covered  in  the  survey.  Moreover,  the 
description  of  the  cleaning  of  the  fishing  data  in  the  TER  Report  omits  several  pertinent  details, 
such  as  the  number  of  respondents  eliminated  during  the  cleaning  process.' 

Furthermore,  some  of  the  information  in  the  written  materials  conflicts  with  other  written 
information  or  with  information  in  the  computer  files.  For  example,  I  have  an  RTI  Memorandum 
written  by  Sharon  Snow  to  Sara  Hudson  and  others  on  June  13,  1994  (TER000033384-89)  on 
the  subject  of  "Final  data  cleaning  activifies  for  Montana  Outdoor  Recreation  Survey"  which 
describes  data  cleaning  for  the  last  batches  of  survey  data,  consisting  of  survey  booklets  from  the 
final  wave  of  the  survey  (July  -  August,  1993),  as  well  as  late  returns  of  survey  booklets  from 
earlier  waves  all  the  way  back  to  the  first  wave  (July  -  August,  1992).  However,  the  number  of 
fishing  trips  that  remains  after  cleaning  as  reported  in  this  memorandum  is  substantially  different 
from  the  number  of  fishing  trips  used  to  estimate  the  TER  model  and  contained  in  the  computer 
file  provided  to  me.  There  are  several  other  differences  in  details  of  the  cleaning  process  as 
described  in  the  June  13  Memorandum  versus  the  TER  Report. 

In  short,  at  this  time  I  do  not  have  adequate  access  to  the  data  generated  by  the  MOR 
panel  survey  to  fully  verify  or  reproduce  the  analysis  in  the  TER  Report.  Therefore,  at  this  time 
I  can  provide  only  a  partial  assessment  of  the  MOR  panel  survey. 

The  TER  Report  mentions  that  two  of  the  1 149  people  who  agreed  to  participate  in  the 
MOR  panel  survey  withdrew  from  it  before  the  first  booklet  was  mailed,  and  another  171  panel 
members  either  moved  out  of  state  or  withdrew  from  the  survey  for  a  variety  of  reasons 
(including  a  few  who  said  they  were  concerned  about  the  sponsor  of  the  survey).  For  the 
remaining  participants,  the  TER  Report  states  (p.  18)  that  the  average  response  rate  for  returned 
booklets  over  the  seven  mailings  between  Wave  1  (July-August  1992)  and  Wave  7  (July-August 
1993)  was  nearly  73%.  Since  I  do  not  have  access  to  the  actual  data  on  who  dropped  out  of  the 
panel,  I  will  assume  that  this  is  true.  However,  it  is  misleading  because  it  apparently  includes 
individuals  who  turned  in  blank  or  incomplete  booklets. 


From  the  information  provided,  the  cleaning  seems  to  have  been  somewhat  heavy-handed.  For  example,  almost 
5%  of  the  fishing  trips  were  dropped  because  the  respondent  provided  no  income  data,  even  though  income  was  not 
an  explicit  variable  in  the  model.  In  such  cases  of  missing  data,  imputation  would  have  been  an  alternative  to 
dropping  observations. 
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If  one  focuses  on  fishing  ~  which  is  the  only  recreation  activity  the  TER  Report  considers 
~  while  there  were  881  people  who  identified  themselves  as  having  fished  over  the  past  year 
among  the  members  of  the  MOR  panel,  only  303  panel  members  reported  participating  in  fishing 
during  the  14  months  of  the  MOR  survey.^  This  implies  a  reporting  rate  of  only  2>AA%.  Two 
quahfications  should  be  noted.  173  members  dropped  out  the  panel  during  the  course  of  the 
survey,  some  of  whom  were  anglers,  which  would  raise  the  rate  of  reporting  to  some  degree;  and 
some  respondents  who  reported  fishing  trips  were  probably  eliminated  during  the  cleaning  of  the 
data.  Even  with  these  qualifications,  it  is  remarkable  that  only  303  of  the  surviving  976  panel 
remembers  reported  participating  in  fishing  during  the  course  of  the  MOR  survey. 

I  believe  that  some  of  this  is  an  artefact  of  the  MOR  survey.  I  noted  earlier  the  failure 
of  the  telephone  survey  to  produce  results  consistent  with  either  the  Census  Bureau's  National 
Survey  on  FHWAR  or  the  Montana  DFWP  Statewide  Angling  Pressure  Survey.  It  appears  that 
the  MOR  telephone  survey  overestimated  the  number  of  anglers  among  adult  Montana  residents 
by  about  74%.  Therefore,  instead  of  881  anglers  among  the  1 149  members  of  the  MOR  panel, 
there  may  have  been  only  about  505  anglers  in  the  panel.  If  the  people  that  withdrew  from  the 
panel  were  evenly  distributed  among  anglers  and  non-anglers,  this  would  reduce  the  number  of 
anglers  surviving  in  the  panel  to  about  430.  Having  303  individuals  report  on  fishing  trips  would 
then  constitute  a  reporting  rate  of  about  70%. 

This  does  not  mean,  however,  that  there  were  no  significant  reporting  problems  with  the 
MOR  panel  survey.  While  there  were  303  anglers  among  the  respondents,  not  all  of  them  seem 
to  have  reported  participation  in  fishing  consistently  throughout  the  course  of  the  survey.  Table 
3  presents  the  data  on  reporting  of  fishing  participation.  July  is  the  peak  month  for  fishing  in 
Montana  and,  indeed,  July  1992  is  the  peak  month  for  the  reporting  of  fishing  trips  in  the  MOR 
survey.  Thereafter,  however,  the  data  show  a  dramatic  decline  in  the  reporting  of  fishing 
participation  during  the  course  of  the  survey.  Of  224  individuals  who  reported  participating  in 
fishing  during  the  first  wave  of  the  survey  (July- August  1992),  only  64  reported  participating  in 
fishing  during  the  last  wave  (July-August  1993).  It  appears  that  nearly  three-quarters  of  the 
anglers  in  the  panel  at  the  beginning  of  the  MOR  survey  had  stopped  reporting  their  participation 
in  fishing  by  the  end. 

Was  the  attrition  of  the  MOR  panel  really  as  drastic  as  it  seems?  I  can  think  of  three  other 
possible  explanations,  but  it  appears  that  they  can  be  discounted.  Overestimation  of  the  number 
of  anglers  in  the  population  as  a  result  of  errors  in  the  MOR  telephone  survey  is  unlikely  to  be 
a  factor  here  because  it  would  affect  all  waves  alike.  The  elimination  of  respondents  during  data 
cleaning  could  be  a  factor  but,  if  anything,  I  would  expect  it  to  work  in  the  opposite  direction  - 
-  according  to  the  June  13,  1994  Memorandum  by  Sharon  Snow,  fewer  observations  were 
eUminated  during  data  cleaning  for  wave  7  than  wave  1  because  there  were  fewer  incomplete  or 
anomalous  responses.  Therefore,  data  cleaning  is  unlikely  to  have  contributed  to  the  appearance 
of  sample  attrition.  A  third  possibility  is  that  there  really  was  a  dramatic  change  of  heart  among 


One  of  these  has  a  zip  code  outside  Montana. 
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adult  residents  of  Montana  at  the  end  of  1992  such  that  the  vast  majority  of  anglers  suddenly  lost 
interest  in  fishing  in  the  state.  Under  this  hypothesis,  the  change  in  fishing  participation  is  not 
a  survey  artefact  but  a  real  phenomenon.  However,  there  is  no  evidence  to  corroborate  this. 

That  leaves  survey  attrition  and  what  Bailar  called  time-in-survey  bias  as  the  prime 
explanations  for  the  huge  reduction  in  the  reporting  of  fishing  participation  in  the  MOR  panel 
survey.  How  likely  are  these  to  have  occurred?  As  indicated  by  the  quotation  from  Sudman  and 
Ferber  (1979),  it  is  not  uncommon  for  many  participants  to  drop  out  of  a  panel  survey  after  two 
or  three  waves.  Having  three-quarters  drop  out  over  12  months  is  fairly  extreme,  but  by  no  means 
unknown.  To  counteract  this,  survey  designers  adopt  a  strategy  of  replenishing  a  panel  by 
recruiting  new  members  every  few  waves.  In  effect,  they  create  a  rotating  panel.  This  ensures 
that,  at  any  one  time,  not  all  the  panel  members  have  been  in  the  panel  for  the  same  length  of 
time  and  therefore  are  not  all  experiencing  time-  and  attrition-related  effects  to  the  same  degree. 
This  permits  one  to  identify  and  statistically  correct  for  time-in-survey  bias.  Replenishing  a  panel 
is  obviously  more  expensive  than  a  single  recruitment,  but  it  provides  superior  protection  against 
attrition  phenomena.' 

While  attrition  is  a  threat  in  any  panel  survey,  in  the  case  of  the  MOR  survey  there  were 
two  additional  factors  which  made  it  especially  likely.  The  first  is  that  participants  were  paid 
their  participation  fee  of  $5  per  wave  regardless  of  whether  they  returned  a  booklet  or  not,  and 
regardless  of  how  carefully  they  completed  it.  While  RTI  engaged  in  considerable  effort  to  chase 
down  panel  members  who  failed  to  turn  in  a  booklet,  it  seems  to  have  been  less  concerned  about 
completeness  of  recording  for  those  who  did  turn  in  a  booklet.  The  second  factor  is  that  this 
survey  was  relatively  onerous  for  respondents.  The  booklet  itself  is  a  lengthy  and  forbidding 
document,  consisting  of  9  separate  sections.  Even  the  short  version  in  the  TER  Report  runs  to 
28  pages.  Respondents  are  asked  to  record  details  of  every  trip  for  undertaken  for  the  primary 
purpose  of  fishing,  boating  (motor  and  sail-boating,  canoeing,  kayaking,  rafting,  jet  skiing,  row 
boating,  tubing,  windsurfing,  water  skiing,  swimming  in  lakes,  rivers,  etc.),  tent  camping  and  RV 
or  trailer  camping,  hiking  (day  hiking/walking,  backpacking,  horseback  riding,  mountain  biking), 
large  and  small  game  hunting,  duck  and  geese  hunting,  recreational  trapping,  and  other  activities 
(picnicking/general  park  use,  wildlife  and  bird  watching,  wildlife  and  landscape  photography, 
rock  climbing,  horseback  riding,  sight-seeing).  For  each  individual  trip,  20  or  more  questions  had 
to  be  answered.  For  trips  with  multiple  destinations,  54  questions  had  to  be  answered.  All  of  this 
was  to  be  done  with  pencil  and  paper  in  a  self-administered  survey.  It  was  done  not  for  some  of 
the  outdoor  recreation  trips  over  a  period  of  14  months,  but  for  all  of  them.  From  what  I  know 
of  some  people  in  Montana,  answering  the  questionnaire  would  itself  have  been  a  full-time  job. 
It  would  be  like  asking  me  to  account  for  every  cookie  that  I  eat  over  a  period  of  14  months.  For 
my  part,  I  would  rather  spend  the  time  eating  than  recording!  Given  the  tedious  nature  of  the  RTI 
questionnaire,  it  would  not  be  surprising  if  some  respondents  chose  to  fill  it  in  selectively, 
skipping  some  trips  and  reporting  others  incompletely.  Indeed,  it  would  be  a  modem  miracle  if 


'  Recruiting  a  new  panel  each  wave,  as  in  the  Montana  DFWP  Angling  Pressure  Surveys,  is  the  ultimate  form 
of  panel  replenishment.  This  reduces  attrition-related  phenomena  to  the  minimum,  but  requires  the  most  effort  for 
panel  recruitment. 
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this  had  not  happened. 

In  summary,  RTI  chose  to  administer  an  extensive  and  burdensome  questionnaire  to  a 
panel  of  respondents  for  a  relatively  lengthy  period  of  time,  without  replenishing  the  panel  during 
the  course  of  the  survey.  Perhaps  this  was  done  to  save  money.  Whatever  the  reason,  the  results 
appear  to  have  been  fairly  calamitous. 

The  same  conclusion  arises  if  one  considers  the  temporal  distribution  of  fishing  trips. 
Table  4  shows  the  distribution  of  the  fishing  trips  reported  by  the  303  anglers  in  the  MOR  panel 
over  the  survey  waves.  For  comparison,  it  also  shows  the  distribution  of  fishing  trips  for  the  same 
time  of  year  reported  by  45,384  resident  anglers  in  the  1991  Montana  DFWP  Angling  Pressure 
Survey  (McFarland  and  Hughes  1994,  Table  5).*  These  data  are  graphed  in  Figure  1.  For  both 
surveys,  Figure  1  normalizes  the  number  of  visits  in  July- August  to  100,  and  then  traces  out  the 
number  of  visits  in  each  other  wave  relative  to  this.  The  graph  covers  14  months,  starting  and 
ending  with  July-August.'  July  is  the  peak  month  for  fishing  by  resident  anglers  in  Montana, 
followed  by  August.  After  September,  fishing  declines  considerably  as  winter  approaches. 
November  is  the  low  month  for  fishing.  In  January,  after  the  holidays,  winter  fishing  reaches  its 
peak.  In  April,  fishing  begins  to  pick  up  and,  by  May,  the  summer  fishing  season  is  under  way. 
It  is  not  surprising,  therefore,  that  the  number  of  fishing  trips  reported  by  members  of  the  MOR 
panel  declines  in  wave  2  (September-October)  and  reaches  its  low  in  wave  3  (November- 
December).  As  with  the  DFWP  data,  a  winter  peak  occurs  in  wave  4  (January-February).  But 
then,  the  two  curves  begin  to  diverge.  In  wave  5  (March-April),  trips  reported  in  the  MOR  survey 
fall  off  relative  to  the  DFWP  data.  This  is  more  pronounced  in  wave  6  (May-June).  In  wave  7 
(July-August),  the  non-reporting  in  the  MOR  panel  is  so  pronounced  that  there  are  actually  fewer 
trips  than  in  wave  6,  even  though  it  is  now  the  height  of  summer.  This  is  a  stark  depiction  of  the 
time-in-survey  bias  associated  with  the  MOR  panel  survey. 

Not  only  is  the  non-reporting  in  the  MOR  survey  quite  extreme,  but  also  it  does  not  occur 
at  random.  For  example,  among  respondents  those  who  reported  fishing  in  wave  1,  those  who 
said  in  the  telephone  survey  that  they  participated  frequently  in  fishing  were  significantly  more 
likely  to  continue  to  report  on  their  fishing  through  wave  7  than  those  who  identified  themselves 
as  participating  sometimes  or  seldom:  34%  of  the  former  reported  on  their  fishing  in  wave  7, 
compared  to  20%  of  the  latter.  Similarly,  those  who  took  4  or  more  trips  in  wave  1  were 
significantly  more  likely  to  continue  reporting  through  wave  7  than  those  who  took  1  -  3  trips 
in  wave  1;  55%  of  the  former  reported  on  their  fishing  in  wave  7,  compared  to  23%  of  the  latter. 
Thus,  it  was  the  more  active  anglers  who  kept  up  their  reporting  of  fishing. 

The  TER  researchers  were  not  unaware  of  the  decline  in  the  reporting  of  fishing  trips  as 
the  MOR  survey  progressed.  What  did  they  do  about  it?  It  appears  that  they  placed  their  faith 


*  The  results  of  the  1993  Survey  are  not  yet  available  to  me. 

'  Since  the  Montana  DFWP  data  cover  only  12  months,  I  use  the  same  data  for  July- August  at  the  beginning 
and  the  end  of  the  graph. 
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TABLE  4:  BREAKDOWN  OF  RESIDENT  FISHING  IN  MONTANA  BY  WAVE 


WAVE 


RTI  -  MOR  DFWP  ANGLING 

SURVEY  1 2  -  MONTH    PRESSURE  SURVEY  1 2  -  MONTH 
(1992-93)   SHARES         (1991-92)      SHARES 
(1)  (2) 


1  July  -  August 

2  September  -  October 

3  November  -  December 

4  January  -  February 

5  March -April 

6  May -June 

7  July -August 


538 
283 
116 
171 
138 
271 
219 


35.5% 
18.7% 

7.6% 
11.3% 

9.1% 
17.9% 


656,018 

36.3% 

256,561 

14.2% 

100,106 

5.5% 

155,096 

8.6% 

175,861 

9.7% 

463,803 

25.7% 

TOTAL:  WAVES  1  -  6  1,517 

TOTAL:  WAVES  1  -  7  1.736 


100.0% 


1,807,445 


100.0% 


SOURCE:  (1)  Fishing  trips,  calculated  from  MOR  data 

(2)  Days  of  fishing  from  McFariand  (1994a),  Table  5 
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in  re-weighting  the  data.  As  panel  members  dropped  out  of  the  survey,  the  weights  on  the 
remaining  members  were  correspondingly  increased.  Evidently  the  TER  researchers  hoped  that 
this  would  offset  the  shrinkage  in  panel  reporting. 

To  assess  how  well  this  solved  the  problem,  I  reviewed  the  weighted  estimates  of 
participation  in  fishing  by  adults  residents  of  Montana  that  are  presented  in  the  TER  report  and 
compared  them  with  estimates  from  the  National  Survey  of  FHWAR  and  the  Montana  DFWP 
Statewide  Angling  Pressure  Survey.  The  results  are  shown  in  Table  5.  Since  these  other  data  are 
developed  on  a  12-  rather  than  14-month  basis,  I  consider  the  weighted  estimates  of  fishing 
participation  from  the  MOR  survey  for  the  period  from  July  1992  through  June  1993. 

Using  its  weights,  the  TER  Report  estimates  that  adult  Montana  residents  took  682,603 
fishing  trips  during  this  period  to  the  fishing  sites  covered  by  its  fishing  demand  model.  The 
model  is  based  on  what  is  known  as  the  random  utility  maximization  (RUM)  hypothesis,  and  I 
will  refer  to  the  sites  in  the  model  as  the  RUM  sites.  There  are  97  of  these  sites,  covering  most 
of  western  and  central  Montana.  By  virtue  of  their  location  in  the  mountains  rather  than  in  the 
eastern  plains,  they  account  for  the  great  majority  of  the  fishing  in  Montana.  The  Census  Bureau 
estimates  that  adult  Montana  residents  took  1,557,700  in-state  fishing  trips  in  1991.  The  Census 
Bureau  does  not  break  its  estimate  down  by  region,  but  this  is  clearly  higher  than,  and 
inconsistent  with,  the  estimate  in  the  TER  Report. 

With  respect  to  trip  length,  according  to  the  TER  Report  the  average  length  of  fishing 
trips  reported  in  the  MOR  survey  was  1.45  days  per  trip.  The  Census  Bureau  reports  that  the 
average  in-state  fishing  trip  by  Montana  residents  lasted  1.2  days.  This  suggests  that  the  MOR 
survey  systematically  over-sampled  longer  trips. 

Combining  trip  length  with  number  of  trips  yields  an  overall  estimate  of  989,774  days  of 
fishing  by  adult  Montana  residents  at  the  RUM  sites  in  the  TER  Report.  The  Census  Bureau 
estimates  that  adult  Montana  residents  engaged  in  1,872,000  days  of  fishing  for  the  state  as  a 
whole.  The  1991  Montana  DFWP  Statewide  Angling  Pressure  Survey  estimates  that  Montana 
residents  engaged  in  1,807,448  days  of  fishing  in-state.  This  corresponds  very  closely  with  the 
Census  Bureau  estimate;  it  lies  well  within  the  Census  Bureau's  90%  confidence  interval.  The 
Angling  Pressure  Survey  breaks  down  its  estimate  by  individual  fishing  areas.  I  have  identified 
the  areas  that  correspond  to  the  RUM  sites,  and  estimate  that  they  account  for  about  1,591,800 
(88%)  of  the  1,807.448  days  of  resident  fishing  in  the  1991  Angling  Pressure  Survey.  This 
suggests  that  TER's  weighted  estimate  of  fishing  activity  based  on  the  MOR  survey 
underestimates  the  true  amount  of  fishing  at  the  RUM  sites  by  over  a  third  (989,774  days  vs 
1,591,834  days).  If  one  assumes  that  the  RUM  sites  account  for  same  proportion  of  trips  by 
residents  anglers  to  sites  in  Montana  as  days  of  resident  fishing,  and  applies  this  proportion  to 
the  Census  Bureau  estimate  of  the  number  of  resident  trips  statewide,  it  suggests  that  there  were 
about  1,371,850  trips  by  residents  to  the  RUM  sites  in  1991.  Hence,  TER's  weighted  estimate 
of  682,603  trips  annually  to  the  RUM  sites  is  almost  exactly  half  of  the  true  total  (49.8%). 

To  summarize,  it  appears  that  there  are  two  biases  in  the  weighted  estimates  of  fishing 
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TABLE  5:    IN-STATE  FISHING  BY  MONTTANA  RESIDENT  ANGLERS 


U.S.  CENSUS 

RTI  -  MOR 

DFWP  ANGLING 

BUREAU 

SURVEY 

PRESSURE  SURVEY 

(1991) 

(1992-93) 

(1991-92) 

(12mos) 

(12mos) 

(12mos) 

FISHING  TRIPS  (in  thousands) 


ALL  SITES  1,557.7 

RUM  SITES  1,371.9   "  682.6 


DAYS  PER  FISHING  TRIP 

ALL  SITES  1.20 

RUM  SITES  1.45 


DAYS  OF  FISHING  (in  thousands) 

ALL  SITES  1,672.0  1,807.4 

RUM  SITES  989.8  1,591.8 


SOURCE:    1991  National  Survey  of  FHWAR:  Montana,  Table  3 
TER  (1995),  Table  6-2 
McFarland  (1994a),  Table  6 
**  Estimated  assuming  proportion  from  DFWP  Survey 


activity  based  on  the  MOR  survey:  an  under-reporting  of  fistiing  generally,  and  an  oversampling 
of  longer  trips.  While  the  biases  go  in  opposite  directions,  the  former  predominates  and  produces 
a  gross  undercount  of  fishing  activity  at  the  RUM  sites  in  the  TER  Report.  I  conclude  that  the 
MOR  panel  survey  did  not  meet  its  goal  of  providing  a  representative  sample  of  fishing  trips  by 
the  target  population  of  Montana  adult  resident  anglers.  Moreover,  the  weighting  procedures  used 
by  TER  failed  to  correct  for  the  errors  in  survey  execution. 

There  are  several  reasons  why  the  weights  used  by  TER  failed  so  miserably.  First,  like 
everything  else  in  the  MOR  survey,  the  weights  appear  to  have  been  designed  for  a  broad  range  - 
-  an  extremely  broad  range  —  of  outdoor  recreation  activities,  not  just  fishing.  TER's  decision  to 
limit  its  analysis  to  fishing  must  have  come  occurred  relatively  late  in  the  analysis  phase,  after 
the  weights  had  been  determined,  because  the  TER  weights  are  keyed  to  the  returning  of  sur\'ey 
booklets  in  general,  not  to  the  reporting  of  fishing.  In  wave  7,  for  example,  the  TER  weights 
reflect  the  fact  that  only  666  panel  members  turned  in  booklets  for  that  wave  compared  to  854 
for  wave  1,  rather  than  the  fact  that  only  84  of  them  reported  fishing  in  wave  7  compared  to  224 
in  wave  1.  This  is  surely  a  mistake  for  a  study  that  now  focuses  exclusively  on  fishing. 

Second,  the  failure  of  the  weights  in  the  panel  survey  illustrates  what  I  said  in  connection 
with  the  MOR  telephone  survey  that  weighting  is  not  a  panacea  and  that  use  of  inappropriate 
weights  can  be  counterproductive.  As  with  the  telephone  survey,  the  weights  in  the  MOR  panel 
survey  are  benchmarked  against  demographic  characteristics  of  the  general  population  of  Montana 
rather  than  the  population  of  anglers,  and  they  are  designed  for  measuring  population-level 
statistics  rather  than  parameters  of  a  behavioral  model.  An  example  makes  the  point.  Of  the  1737 
fishing  trips  reported  in  the  MOR  survey,  112  trips  (6.5%)  come  from  a  single  angler,  a  63  year 
old  male,  one  of  9  panel  members  from  Kalispell,  who  reported  in  the  telephone  survey  that  he 
fished  sometimes  but  not  frequently.  In  waves  2  and  3,  he  reported  67  fishing  trips,  all  of  them 
to  the  Flathead  River.  As  a  result  of  his  efforts,  90  of  the  1736  fishing  trips  in  the  MOR  data  are 
to  this  site,  giving  it  a  "market  share"  of  5.18%.  By  contrast,  the  Montana  Angling  Pressure 
Survey  for  1991  shows  that  the  site  accounts  for  1.01%  of  trips  by  all  resident  anglers  to  the 
RUM  sites.  Because  this  individual  was  retained  in  the  MOR  data,  the  survey  overestimates  this 
site's  share  of  fishing  trips  by  over  400%.  There  is  one  other  person  who  reported  taking  over 
50  fishing  trips  —  a  59  year  old  male,  one  of  8  panel  members  from  Anaconda,  who  reported  53 
trips,  all  of  them  to  Georgetown  Lake.  As  a  result,  Georgetown  Lake  accounts  for  6.6%  of  the 
fishing  trips  in  the  MOR  survey,  whereas  it  accounts  for  only  2.4%  of  the  trips  to  the  RUM  sites 
in  the  1991  Montana  Angling  Pressure  Survey.  In  this  case,  because  the  individual  was  retained 
in  the  MOR  data,  the  survey  overestimates  the  site's  share  of  fishing  trips  by  175%. 

I  don't  doubt  that  such  anglers  exist  in  the  population.  But,  I  do  believe  that  they  were 
weighted  inappropriately  in  the  MOR  survey.  The  weights  used  by  TER  extrapolate  from  the 
MOR  panel  to  the  general  population  of  Montana.  Consequently,  the  weights  will  produce  the 
right  number  of,  say,  63  year  old  males  in  Kalispell,  or  59  year  old  males  in  Anaconda.  At  the 
same  time,  however,  they  will  imply  that  1/9  of  all  the  anglers  in  Kalispell  take  1 12  fishing  trips, 
63%  of  them  to  the  Flathead  River,  or  that  1/8  of  all  the  anglers  in  Anaconda  take  53  fishing 
trips,  all  to  Georgetown  Lake.  While  yielding  reasonable  population-level  demographics,  the 
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weights  produce  a  highly  distorted  account  of  angler  activity  in  Montana  and  its  break-down 
among  individual  sites  and  time  periods. 

Third,  no  amount  of  weighting  can  correct  for  a  grossly  undersized  survey.  The  MOR 
panel  was  too  small  to  begin  with,  since  it  was  predicated  on  a  telephone  survey  that  seriously 
overstated  the  number  of  anglers  in  the  Montana  population,  and  it  suffered  from  attrition  and 
growing  nonreporting  as  it  progressed.  By  the  last  wave,  in  the  summer  of  1993,  the  TER 
research  team  was  dependent  on  a  sample  of  just  84  anglers  to  represent  the  behavior  of  perhaps 
140,000  anglers  statewide.  I  do  not  believe  that  any  weighting  scheme  could  produce  meaningful 
population-level  estimates  of  fishing  activity  in  those  circumstances.  To  be  sure,  one  can  go 
through  the  motions  of  weighting  up  the  84  anglers.  But,  the  confidence  interval  associated  with 
the  weighted  estimate  will  be  enormous  ~  so  large  as  to  render  it  useless  in  practice.  To  the 
extent  the  intended  application  of  the  MOR  survey  is  to  place  an  economic  value  on  fishing  at 
certain  rivers  in  southwestern  Montana,  the  same  conclusion  applies  with  even  more  force. 


3.5  Conclusions 

The  two-phases  of  the  MOR  survey  constitute  an  unusually  ambitious  and  costly  attempt 
to  survey  outdoor  recreation  on  a  fairly  large  scale,  both  geographically  and  temporally.  However, 
because  of  flaws  in  the  design  and  execution  of  the  survey,  it  employed  an  unrepresentative 
sample  of  Montana  outdoor  recreators  and  it  provides  an  inaccurate  and  unreliable  account  of  in- 
state fishing.  The  MOR  survey  simultaneously  overestimates  the  number  of  adult  resident  anglers 
in  Montana  by  74%  and  underestimates  the  number  of  fishing  trips  they  take  at  the  RUM  sites 
by  50%.  It  overestimates  the  length  of  fishing  trips  by  about  21%.  It  also  provides  estimates  of 
the  allocation  of  these  fishing  trips  among  the  alternative  RUM  sites  that  are  inaccurate,  in  some 
cases  by  more  than  100%. 

These  errors  arose  from  a  combination  of  circumstances.  The  telephone  survey 
substantially  overstated  the  extent  to  which  women  fish  and  hunt  in  Montana,  perhaps  because 
of  deficiencies  in  interviewer  training  and  performance.  For  both  male  and  female  respondents, 
telescoping  may  have  contributed  to  the  overcounting  of  fishing  participation.  Because  the  phone 
survey  greatly  exaggerated  the  number  of  anglers,  the  MOR  panel  actually  contained  far  fewer 
anglers  than  the  TER  research  team  had  anticipated.  I  estimate  that  it  contained  only  about  505 
anglers  ~  much  too  small  a  sample  for  the  ambitious  goal  of  measuring  the  allocation  of  fishing 
effort  among  100  different  locations  over  one  winter  fishing  season  and  two  summer  seasons. 
This  was  compounded  by  tremendous  panel  attrition  and  nonreporting  as  the  survey  proceeded. 
The  main  reason  for  the  attrition  and  nonreporting  was  the  tedious  and  burdensome  nature  of  the 
questionnaire,  combined  with  the  lack  of  incentive  for  complete  reporting.  The  research  team 
failed  to  offset  this  by  replenishing  the  panel  during  the  course  of  the  survey.  The  attrition  and 
nonreporting  did  not  affect  panel  members  at  random;  it  was  the  more  active  anglers  who  kept 
up  their  reporting,  leading  to  a  form  of  avidity  bias  in  the  sampling.  By  the  summer  of  1993,  the 
MOR  survey  was  dependent  on  84  relatively  avid  anglers  to  represent  the  fishing  behavior  of 
about  140,000  anglers  statewide.  The  TER  research  team  relied  on  weighting  to  correct  for  all 
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the  sampling  inadequacies.  In  this  case,  the  faith  in  weighting  was  misplaced.  Indeed,  the 
weighting  seems  to  have  compounded  the  problem  because  since  it  was  keyed  to  the  general 
population  of  Montana  rather  than  the  target  population  of  anglers.  Consequently,  the  weighting 
actually  increased  the  error  in  the  MOR  data. 

Although  the  survey  problems  arose  from  somewhat  subtle  causes,  their  aggregate  effect 
is  gross  and  unmistakable.  To  detect  this,  one  merely  has  to  compare  the  results  of  the  MOR 
survey  with  other  existing  data  on  fishing  in  Montana  such  as  the  Census  Bureau's  National 
Survey  of  FHWAR.  It  doesn't  take  a  rocket  scientist  to  do  this. 

Why  does  the  TER  Report  not  mention  these  obvious  problems  with  the  MOR  survey? 
Is  the  reason  complacency  —  a  belief  that,  since  this  was  the  most  lavish  economic  survey  of 
outdoor  recreation  up  to  now,  its  results  must  be  uniformly  excellent?  Is  the  reason  strategic  — 
if  the  survey  data  are  flawed,  this  destroys  the  credibility  of  the  economic  model  estimated  from 
the  data?  I  suspect  a  bit  of  both.  Whatever  the  reason,  however,  the  failure  to  address  the  flaws 
in  the  MOR  data  is  a  serious  indictment  of  the  TER  research  team. 

Moreover,  it  throws  the  rest  of  their  study  in  jeopardy.  The  central  position  of  the  TER 
Report  is  that  one  can  make  inferences  from  the  sample  in  the  MOR  survey  to  the  whole 
Montana  population.  The  fact  that  the  inferences  produce  results  so  much  at  variance  with  other 
information  on  the  Montana  population  leads  me  to  doubt  this  claim. 
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4.  The  Variables  in  the  RUM  Model 


4.1  Introduction 


This  section  reviews  the  variables  used  in  the  RUM  model  that  appears  in  the  TER 
Report.  The  dependent  variables  in  the  model  are  measures  of  participation  in  fishing  at  the  RUM 
sites;  these  are  discussed  in  Section  4.2.  For  the  purpose  of  placing  an  economic  value  on  fishing, 
the  key  explanatory  variable  is  the  "price"  variable  —  the  cost  that  anglers  associate  with  going 
fishing.  This  is  discussed  in  Section  4.3.  The  other  key  explanatory  variable  needed  to  assess  the 
loss  of  consumer  surplus  from  pollution  in  the  Upper  Clark  Fork  River  Basin  is  one  or  more 
measures  of  how  the  pollution  affects  angler  behavior  and  welfare  ~  i.e,  measures  of  the  aspects 
of  fishing  that  would  affected  by  the  pollution,  such  as  the  quality  of  the  fishing  experience. 
Section  4.4  discusses  this  variable.  The  effects  of  measurement  error  in  this  variable  are 
examined  in  Section  4.5.  Other  site  characteristics  are  considered  in  Section  4.6 


4.2  The  Dependent  Variables 

The  dependent  variables  in  the  RUM  model  are  various  aspects  of  participation  in  fishing, 
including: 

(1)  how  often  the  individual  participates  in  fishing  (number  and  frequency  of  trips) 

(2)  the  length  of  the  fishing  trip,  and 

(3)  the  type  of  trip  (river  vs  lake)  and  the  specific  site  destination. 

Because  of  the  flaws  in  the  MOR  panel  survey,  as  discussed  above,  all  of  these  variables  are 
measured  with  error. 

Compared  to  the  Census  Bureau's  data,  there  is  only  half  the  correct  number  of  trips  by 
resident  anglers  to  the  RUM  sites.  This  biases  the  estimation  of  participation  frequency  and 
renders  it  unreliable.  Similarly,  more  longer  trips  are  reported  in  the  MOR  survey  than  is 
indicated  in  the  Census  Bureau's  data,  which  biases  the  estimation  of  trip  length  and  renders  that 
part  of  the  TER  model  unreliable.  The  breakdown  of  individual  trips  among  sites  in  the  MOR 
is  also  suspect.  I  already  mentioned  the  effects  of  two  outliers,  the  angler  in  Kalispell  who 
reported  112  trips,  90  of  them  to  the  Flathead  River,  and  the  angler  in  Anaconda  who  reported 
53  trips,  all  to  Georgetown  Lake.  Including  these  cases  biases  these  sites'  shares  of  total  trips  in 
the  MOR  data,  and  causes  them  to  diverge  greatly  from  the  shares  in  the  Statewide  Angling 
Pressure  Survey  for  1991.  This  particular  error  can  readily  be  corrected  by  omitting  the  two 
individuals  from  the  data  used  to  estimate  the  model,  but  there  are  other  errors  in  the  site  shares 
that  are  less  readily  corrected.  For  example,  the  breakdown  of  trips  in  the  MOR  data  between 
sites  which  are  rivers  versus  sites  which  are  lakes  differs  significantly  from  what  the  Angling 
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Pressure  Survey  shows  for  the  RUM  sites.  As  shown  in  Table  6,  the  MOR  survey  overestimates 
the  share  of  trips  to  river  sites  in  the  summer,  and  underestimates  it  in  the  winter;  in  both  cases, 
the  difference  is  statistically  significant.  Such  errors  bias  the  estimation  of  site  choice  and  render 
this  part  of  the  TER  model  unreliable. 

Another  worrisome  feature  of  the  site  shares  in  the  MOR  data  is  the  tiny  number  of 
anglers  from  whom  they  are  derived.  As  shown  in  Table  7,  with  one  exception  (an  angler  with 
a  Northern  California  zipcode),  all  the  anglers  in  the  MOR  survey  come  from  one  of  seven 
regions  in  the  state.  In  four  of  the  regions,  there  were  40  or  fewer  anglers  reporting  on  their 
fishing.  These  are  extremely  inadequate  samples  if  one  is  serious  about  modelling  the  choices 
that  anglers  in  these  regions  make  among  almost  100  destinations  in  an  area  about  two-thirds  of 
the  state.  Of  the  members  of  the  MOR  panel,  283  individuals  reported  trips  during  the  summer 
fishing  season  (defined  by  Montana  DFWP  as  May  1  -  September  30),  and  141  reported  trips 
during  the  winter  fishing  season  (October  1  -  April  30).  Twenty  of  the  anglers  in  the  MOR  panel 
(6.6%)  reported  fishing  in  winter  but  not  summer.  In  the  Kalispell  region,  6  out  of  35  anglers 
(17.1%)  reported  fishing  in  winter  but  not  summer.  My  hunch  is  that  the  true  proportion  of 
Montana  residents  who  fish  in  the  winter  but  not  the  summer  is  much  smaller  than  these  figures 
indicate.  If  this  is  so,  the  error  in  the  MOR  data  would  throw  the  seasonal  element  in  the  fishing 
participation  model  out  of  kilter.  Table  7  also  shows  that  121  of  the  283  anglers  in  the  MOR 
survey  who  reported  fishing  in  the  summer  (42.8%)  also  fished  during  the  winter.  I  suspect  that 
the  true  proportion  is  lower  and  that  the  MOR  data  are  an  artefact  resulting  from  over-sampling 
of  avid  anglers  among  those  who  reported. 


4.3  The  Cost  Variable 

The  cost  that  anglers  associate  with  going  fishing  is  crucial  for  placing  an  economic  value 
on  their  fishing  activity.  It  was  the  identification  of  this  cost  that  led  Hotelling  (1949)  to  his 
discovery  of  the  basic  method  for  valuing  recreation  and  similar  nonmarket  activities.  Indeed, 
because  of  its  central  role,  the  method  is  commonly  known  as  the  "travel  cost  method"  (TCM). 

Just  how  one  measures  the  cost  in  practice  is  something  that  economists  have  considered 
since  the  first  applications  of  the  TCM  by  Trice  and  Wood  (1958)  and  Clawson  (1959).  Clawson 
expressed  the  situation  thus:  "  The  costs  of  outdoor  recreation  experiences  are  partly  subjective, 
partly  material.  Some  cash  will  have  to  be  spent  for  travel,  food,  lodging  if  more  than  one  day 
is  involved,  and  for  other  purposes.  The  use  of  equipment  such  as  the  automobile  is  usually 
included  also.  ...  The  time  and  energy  spent  in  travel  will  be  a  cost,  which  may  or  may  not  be 
offset  by  pleasures  from  the  actual  travel  experience."  The  various  categories  of  expenditure  that 
Clawson  mentioned  ~  motor  vehicle  or  other  transportation  cost,  on-site  recreation  costs  (entrance 
fees,  tackle  and  rental  equipment,  etc),  costs  of  food  and  accommodation,  and  time  costs  —  have 
formed  the  basis  for  cost  estimation  in  the  subsequent  TCM  literature. 

There  is  a  fair  amount  of  consensus  in  the  literature  about  how  these  costs  should  be 
estimated.  One  approach  is  to  survey  recreationists  and  ask  what  were  their  expenditures  for 
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TABLE  6:  DISTRIBUTION  OF  TRIPS  BY  RESIDENT  ANGLERS  AMONG  RIVER  AND  LAKE  RUM  SITES 

%  RIVER    %LAKE 

RTl  -  MOR  56.4%        43.6% 

SUMMER 

MOr^TTANA  DFWP  50.5%        49.5% 


RTl  -  MOR  43.7%        56.3% 

WINTER 

MONTANA  DFWP  55.7%        44.3% 


TABLE  7:  NUMBER  OF  ANGLERS  REPORTING  TRIPS  IN  MOR  SURVEY,  BY  REGION 

NUMBER  OF  INDIVIDUALS  REPORTING 
REGION 

BILLINGS  AREA 
GREAT  FALLS  AREA 
NORTH  CENTRAL  AREA 
HELENA  AREA 
SOUTH  WEST  AREA* 
MISSOULA  AREA 
KALISPELLAREA 

OUT  OF  STATE  1  1 

TOTAL  141  283  303 

*  Indudes  Bozeman,  Butte  and  Anaconda 


ZIP  CODES 

WINTER 

SUMMER 

TRIPS 

TRIPS 

EITHER 

590xx,  591XX 

15 

32 

35 

594XX 

18 

39 

40 

595XX 

4 

5 

6 

596XX 

26 

52 

54 

597XX 

33 

63 

68 

598XX 

24 

62 

64 

599XX 

21 

29 

35 

various  items.  The  other  approach  is  to  impute  costs  based  on  information  from  some  external 
source,  for  example  Automobile  Association  data  on  the  costs  of  owning  and  operating  a  vehicle, 
or  what  the  Internal  Revenue  Service  or  other  government  agencies  allow  individuals  for  the  use 
of  their  automobile.  The  main  issue  with  respect  to  automobile  travel  is  whether  or  not  one 
counts  all  of  the  costs.  According  to  the  American  Automobile  Manufacturers  Association,  as 
reported  in  the  Statistical  Abstract  of  the  United  States  1994,  the  average  cost  of  owning  and 
operating  an  automobile  in  the  US  in  1992  was  45.8  cents  per  mile,  broken  down  into  9.2  cents 
per  mile  for  operating  expenses  (gas,  oil,  tires,  and  maintenance),  27.8  cents  per  mile  for 
depreciation,  and  17.6  cents  per  mile  for  other  fixed  costs  (insurance,  license  and  registration,  and 
finance  charge).  In  that  year,  the  IRS  allowed  a  standard  mileage  rate  of  28  cents  per  mile,  while 
the  State  of  Montana  allowed  its  employees  a  mileage  rate  of  27  cents  per  mile;  these  mileage 
allowances  essentially  cover  operating  costs  plus  depreciation.  A  majority  of  the  studies  in  the 
TCM  literature  that  impute  a  transportation  cost  use  something  like  the  government  mileage 
rate.'"  However,  some  studies  exclude  depreciation  on  the  grounds  that  this  is  a  fixed  expense 
which  should  not  influence  people's  choice  of  destination  for  recreation.  Of  course,  whether  or 
not  it  does  influence  their  choices  is  an  empirical  question  that  could  be  resolved,  for  example, 
by  questioning  them." 

The  MOR  panel  survey  did  request  respondents  to  report  what  they  saw  as  transportation 
and  other  costs.  However,  the  TER  research  team  chose  not  to  use  the  survey  responses.  Instead, 
it  imputed  a  cost  based  on  respondents'  reported  miles  per  gallon  combined  with  an  external 
estimate  of  the  cost  of  gasoline  per  gallon  and  the  cost  of  oil,  tires,  and  maintenance.  The 
gasoline  cost  averages  about  6  cents  per  mile,  and  the  other  costs  are  imputed  at  5  cents  per  mile, 
for  a  total  transportation  cost  in  the  TER  Report  averaging  about  1 1  cents  per  mile.  I  assume  that 
this  is  lower  than  what  the  survey  responses  show. 

With  regard  to  monetary  expenditures  for  non-transportation  items,  the  TER  research  team 
excludes  these  entirely.  This  is  a  relatively  extreme  position.  Most  other  TCM  researchers  include 
some  of  the  on-site  recreation  costs  or  food  and  lodging  costs,  especially  for  longer  trips.  Indeed, 
I  do  not  know  of  any  other  study  in  the  literature  which  has  modelled  trip  length  as  a  endogenous 
choice  variable  while  excluding  from  the  analysis  costs  such  as  food  or  lodging  that  might  be 
expected  to  vary  with  trip  length. 

With  regard  to  time  cost  of  recreation,  the  TER  research  team  include  travel  time,  valued 
at  one  third  of  the  wage  rate,  but  exclude  time  spent  on-site.  It  is  not  uncommon  in  the  literature 
to  ignore  on-site  time  when  dealing  with  short  trips.  When  dealing  with  long  trips  and  the  choice 
of  trip  length,  however,  this  makes  less  sense  because  the  time  commitment  is  likely  to  be  a 


For  example,  this  is  true  of  most  of  the  studies  listed  in  Desvousges  et  al.  (1986),  Table  2-1. 

People  may  decide  to  buy  a  particular  type  of  vehicle  for  their  second  car  precisely  because  it  will  be  more 
useful  for  the  type  of  outdoor  recreation  they  like.  If  so,  the  fixed  costs  of  vehicle  ownership  are  directly  attributable 
to  their  recreation  activity.  The  fact  that  respondents  in  the  MOR  survey  usually  take  a  truck  to  go  fishing  (see  Table 
6.3)  is  certainly  consistent  with  the  argument  that  fixed  costs  should  be  counted. 
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significant  component  of  the  cost  associated  witli  taking  longer  trips. 

There  is  disagreement  in  the  literature  with  regard  to  the  valuation  of  travel  time.  Some 
researchers  impute  a  value  of  time  whereas  others  estimate  it  from  the  data.  Those  who  impute 
a  value  of  time  typically  make  it  some  specified  fraction  of  the  wage  rate.  Cesario  (1976) 
suggested  valuing  travel  time  at  one  third  of  the  wage  rate,  based  on  findings  in  the  literature 
dealing  the  choice  of  transportation  mode  when  commuting  to  work.  Others  have  suggested  using 
a  higher  fraction.  The  TER  Report  seems  to  imply  that  one  third  of  the  wage  rate  is  some  sort 
of  upper  bound  on  the  opportunity  cost  of  recreation  travel  time:  "because  recreation  travel  is 
different  than  commuting  to  work,  it  is  reasonable  that  the  opportunity  cost  of  recreation  travel 
time  is  lower  than  that  associated  with  commuting."  This  is  not  a  position  that  Dr.  Desvousges 
has  always  espoused.  Earlier  in  his  career  when  he  was  conducting  research  aimed  at  publication 
in  academic  journals,  rather  than  intended  for  use  in  litigation,  he  took  the  position  that  the  effect 
of  time  constraints  on  choices  by  recreationists  is  a  complex  matter  which  will  vary  with 
circumstances  and  that  "simple  approximations  to  relate  the  opportunity  cost  of  time  to  the  wage 
rate  will  not  be  able  to  accommodate  all  applications"  (Smith,  Desvousges  and  McGivney,  1983). 
However,  given  that  one  needs  to  use  some  such  approximation,  while  Cesario' s  proposal  to 
value  time  at  one  third  of  the  wage  rate  sometimes  fits  the  data  well,  at  other  times  one  gets  a 
better  fit  by  assuming  that  travel  time  is  valued  at  100%  of  the  wage  rate:  "Cesario' s  proposal 
is  not  unambiguously  superior  to  using  the  wage  rate  as  the  opportunity  cost.  Indeed,  until  better 
information  on  the  nature  of  time  constraints  facing  individuals  in  their  recreation  decisions  is 
available,  the  wage  rate  provides  an  equally  plausible  approximation  for  the  opportunity  cost  of 
travel  time"  {op.  cit). 

I  believe  that  Dr.  Desvousges'  earlier  position  is  considerably  more  reasonable  than  the 
one  he  now  adopts.  In  my  own  research  I  have  shown  that,  in  the  face  of  constraints  on  one's 
time,  the  opportunity  cost  of  time  for  recreation  can  substantially  exceed  the  wage  rate  and  need 
not  be  bounded  by  the  opportunity  cost  of  time  spent  commuting  to  work.  Therefore  I  certainly 
cannot  regard  valuing  travel  time  at  one  third  of  the  wage  rate  as  a  conservative  assumption. 

In  summary,  the  TER  research  team  have  chosen  ways  to  measure  the  travel  cost  variable 
that  are  likely  to  bias  it  downwards  to  some  degree.  Their  costing  of  automobile  travel  at  1 1 
cents  per  mile,  while  within  the  range  encompassed  by  the  existing  literature,  is  not  the  majority 
position.  Their  exclusion  of  all  other  monetary  costs  is  generally  inconsistent  with  the  normal 
practice  in  the  literature  for  the  type  of  recreation  trips  involved.  Finally,  as  indicated  above,  their 
valuation  of  the  opportunity  cost  of  travel  time  at  one  third  of  the  wage  rate  is  by  no  means  a 
conservative  assumption. 

Without  having  access  to  the  full  data  from  the  panel  survey,  I  am  not  in  a  position  to 
verify  the  computation  of  the  travel  cost  variable  in  the  RUM  model  or  to  assess  the  likely 
magnitude  of  its  bias.  Based  on  the  information  available  to  me  at  this  time,  my  rough  guess 
would  be  that  the  costs  are  underestimated  by  a  factor  of  two  for  short  trips,  and  considerably 
more  for  long  trips. 
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Why  does  this  matter?  Is  the  measurement  of  the  travel  cost  variable  merely  an  arcane 
academic  nicety,  or  does  it  have  some  practical  implications? 

Obviously,  correct  measurement  of  variables  is  important  for  the  accuracy  and  reliability 
of  a  statistical  model.  However,  it  is  especially  important  in  the  case  of  the  travel  cost  variable 
because  this  serves  to  scale  all  economic  values  derived  from  the  recreation  demand  model. 
Hence,  scaling  errors  in  the  measurement  of  travel  cost  translate  directly  into  scaling  errors  in 
the  measurement  of  economic  value.  The  result  can  be  demonstrated  mathematically,  but  it  also 
has  an  intuitive  explanation.  Suppose  the  travel  costs  are  underestimated  by  a  common  scale 
factor.  As  a  specific  example,  suppose  they  are  underestimated  by  a  factor  of  two  ~  for  each 
person  in  the  sample,  his  travel  cost  for  every  site  is  mistakenly  halved.  His  true  cost  is,  say, 
$6.00  per  trip  for  site  A  and  $8.40  per  trip  for  site  B,  but  in  the  data  from  which  the  model  is 
estimated  these  costs  are  erroneously  recorded  as  $3.00  and  $4.20,  respectively.  From  the  point 
of  view  of  the  computer  estimating  the  model,  it  is  as  though  there  has  been  a  change  of 
monetary  units  from  units  of  $1  to  "new"  units  of  $2.  It  sees  the  prices  of  the  sites  in  terms  of 
the  new  units  as  3  and  4.2,  respectively.  Consequently,  when  the  coefficients  of  the  model  are 
estimated,  they  are  to  be  interpreted  in  terms  of  the  "new"  units.  So  are  estimates  of  consumer's 
surplus  derived  from  the  fitted  model.  Hence,  if  costs  are  underestimated  by  50%,  so  are  dollar 
estimates  of  consumer's  surplus.  Thus,  to  the  extent  that  travel  costs  are  biased  downwards  by 
a  common  factor  in  the  TER  model,  the  estimates  of  lost  consumer's  surplus  in  the  Upper  Clark 
Fork  River  Basin  will  be  biased  downwards  by  same  proportion. 


4.4  Site  Characteristics 

In  addition  to  travel  cost,  the  RUM  model  uses  several  other  variables  to  measure  fishing 
site  characteristics  ~  aspects  of  sites  that  might  give  utility  (or  disutility)  to  anglers  and  influence 
their  choice  of  destination.  Section  C.2  in  the  TER  Report  gives  a  long  list  of  site  characteristics 
that  were  considered  for  the  RUM  model.  The  great  majority  of  them  seem  to  have  vanished 
without  trace  ~  they  are  not  used  in  the  model  as  reported,  nor  is  there  any  discussion  of  the 
reasons  why  they  were  not  used  or  how  their  inclusion  would  affect  the  coefficients  of  the 
variables  that  are  included. 

The  model  presented  in  the  TER  Report  employs  seven  site  characteristic  variables  in 
addition  to  travel  cost.  Four  are  common  to  choices  among  both  river  and  lake  sites;  three  vary 
by  choice  model.  The  common  characteristics  are  a  dummy  variable  for  whether  the  site  is 
identified  as  a  major  fishing  site  in  The  Angler's  Guide  to  Montana,  the  log  of  the  area  of  the 
site  as  measured  in  USGS  map  blocks,  the  number  of  campgrounds  at  the  site  per  USGS  block, 
and  the  number  of  State  Recreation  Access  (SRA)  areas  at  the  site  per  USGS  block.  While 
acreage  of  area  may  be  a  good  measure  of  size  for  land-based  activities  such  as  hiking  and 
camping,  it  seems  less  appropriate  for  fishing  at  water  bodies.  For  fishing,  the  amount  of 
shoreline  may  be  a  better  measure  of  size  than  simple  area.  Further,  while  crowding  may  be  an 
attribute  that  is  important  to  anglers,  there  may  be  other  variables  that  capture  it  better  than  area. 
With  regard  to  using  the  number  of  campgrounds  or  SRA  areas  as  measures  of  site  quality,  there 
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could  be  some  endogeneity:  the  establishment  of  campgrounds  at  a  site,  for  example,  could 
reflect  the  fact  that  this  is  an  attractive  site  —  it  is  a  consequence  rather  than  a  cause  of  what  one 
seeks  to  measure.  Similar  with  the  dummy  variable  for  whether  a  guide  book  rates  this  as  a  major 
site.  This  tells  you  that  the  site  is  popular  without  saying  why.  It  may  be  highly  correlated  with 
the  dependent  variable  not  because  it  explains  visitation  in  some  legitimate  manner  but  rather 
because  it  is  essentially  another  measure  of  visitation. 

In  the  case  of  the  choice  model  for  river  fishing,  the  other  three  site  characteristics  are 
the  number  of  restricted  species  at  the  site,  the  biomass  of  trout  measured  in  pounds  per  1 ,000 
feet  of  river,  and  a  dummy  variable  for  the  aesthetics  of  fishing  at  the  site.  In  the  choice  model 
for  lake  fishing,  the  variables  are  a  dummy  variable  for  lakes  with  good  access,  a  dummy 
variable  for  lakes  with  abundant  fish,  and  a  dummy  variable  for  the  aesthetics  of  fishing  at  the 
lake  site.  Of  these,  the  biomass  variable  is  the  crucial  one,  given  the  intended  application  of  the 
model.  The  variable  has  two  roles.  First,  it  contributes  to  the  model's  explanatory  power  with 
respect  to  the  allocation  of  fishing  trips  —  it  helps  explain  why  an  angler  chooses  one  site  over 
another  when  they  both  have  the  same  travel  cost.  Second,  it  is  the  key  link  for  computing 
economic  damages  from  the  release  of  hazardous  substances  in  the  Upper  Clark  Fork  River 
Basin.  The  damages  from  these  releases  are  incorporated  into  the  RUM  model  as  changes  in  the 
value  of  the  biomass  variable.  If  the  change  in  the  biomass  variable  does  not  adequately  capture 
their  effect  on  anglers,  this  will  bias  the  estimate  of  lost  consumer's  surplus  —  if  it  understates 
the  impact  on  anglers  this  biases  the  damage  downwards,  and  conversely  if  it  overstates  the 
impact  on  anglers.  Similarly,  any  measurement  error  or  statistical  procedure  that  biases  the 
coefficient  estimate  for  the  biomass  variable  will  correspondingly  bias  the  estimate  of  lost 
consumer's  surplus. 

Thus,  the  challenge  for  the  researcher  is  to  identify  an  indicator  of  the  quality  of  fishing 
at  a  site  that  can  be  measured  with  reasonable  accuracy,  captures  the  aspects  of  the  site  that 
motivate  angler  behavior,  and  can  reasonably  be  used  to  represent  the  effects  of  releases  of 
hazardous  substances  in  the  Upper  Clark  Fork  River  Basin. 

For  river  sites,  the  TER  research  team  selected  for  this  purpose  a  variable  from  the 
Montana  Rivers  Information  System  (MRIS),  a  computerized  data  base  maintained  by  the 
Montana  DFWP,  that  purports  to  measure  the  pounds  of  trout  biomass  per  1000  ft.  of  stream 
length.  This  information  is  provided  for  individual  rivers,  streams  and  creeks,  or  designated 
sections  thereof  In  some  cases.  MRIS  gives  no  data  for  a  particular  area;  I  understand  that,  in 
those  cases,  the  TER  research  team  used  some  sort  of  regression  equation  to  impute  missing 
values  (as  far  as  I  can  tell,  this  is  not  discussed  in  the  TER  Report).  In  many  cases,  several 
different  streams  or  sections  of  streams  are  contained  within  a  single  RUM  site.  In  those  cases, 
the  TER  research  team  took  a  weighted  average  of  the  biomass  figures  for  the  individual  areas, 
using  stream  length  for  weights.  This  seems  less  appropriate  than  weighting  on  the  basis  of 
relative  visitation  by  anglers,  or  on  the  basis  of  some  variable  that  captures  angler's  expectations 
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regarding  the  site.'^ 

The  MRIS  data  base  does  not  cover  lakes.  Therefore,  in  place  of  the  MRIS  measure  of 
biomass,  in  the  lake  choice  model  the  TER  research  team  used  a  dummy  variable  taking  the 
value  of  one  for  lakes  with  "abundant"  fish  and  zero  otherwise.  This  was  based  on  information 
from  the  Montana  Interagency  Stream  &  Lake  Database.  The  Report  gives  no  further  information 
about  how  biomass  is  characterized  in  that  database  or  how  the  research  team  converted  it  into 
a  binary-valued  variable.  Using  a  binary-valued  variable  is  a  coarse  way  to  capture  attractiveness 
to  an  angler  of  fishing  at  a  lake  site.  If  one  is  serious  about  modelling  anglers'  choices. of  lake 
fishing,  as  the  TER  Report  claims  to  be,  it  is  hard  to  believe  that  one  would  be  satisfied  with 
such  a  meager  characterization  of  so  important  a  variable.  On  its  face,  therefore,  this  appears  to 
be  an  inadequate  treatment  of  fishing  quality  in  the  modelling  of  choices  among  lakes  relative 
to  what  is  done  for  choices  among  river  sites. 

With  rivers,  however,  although  the  TER  research  team  might  have  thought  they  had  found 
a  good  indicator  of  fishing  quality,  they  eventually  learned  that  this  was  not  so.  They  were 
informed  in  1994  that  the  version  of  the  MRIS  database  which  they  had  used  (Version  2,  issued 
in  1989),  and  especially  its  biomass  data,  was  inaccurate  and  highly  unreliable.  Of  the  records 
in  that  version  with  biomass  data,  62%  had  data  collected  before  1980,  and  70%  had  a  data  with 
a  low  reliabihty  rating.  The  information  on  the  age  and  reliability  of  the  data  had  not  been 
available  in  the  old  version  of  MRIS,  but  it  was  included  in  a  new  version  released  in  February 
1994  (version  3).  In  addition,  the  new  version  incorporated  more  recent  biomass  data  where  this 
had  become  available,  and  it  now  provided  data  by  smaller  and  better  defined  river  segments. 

This  must  have  been  a  devastating  blow  to  the  TER  research  team.'-'  At  least,  it  would 
have  been  for  me  if  I  had  been  in  their  position.  The  key  variable  on  which  one  is  counting  to 
drive  the  site  choice  model  suddenly  turns  out  to  be  grossly  unreliable  and  out  of  date.  The  TER 
Report  states  on  page  30  that  its  objective  was  to  collect  site  data  relevant  to  the  time  period  of 
the  panel  survey  (July  1992  -  August  1993).  Site  attributes  measured  in  the  1970's  hardly  seem 
to  meet  this  requirement. 

I  believe  that  this  problem  was  to  some  extent  one  of  the  TER  researchers'  own  making. 
Most  researchers  in  their  position  would  have  insisted  on  finding  a  site  quality  variable  that  was 
measured  contemporaneously  with  the  recreation  data.  In  my  own  work,  I  have  devoted  enormous 
effort  to  obtaining  site  quality  variables  that  are  both  meaningful  and  pertinent.  In  my  study  of 
beach  recreation  in  Boston,  I  arranged  to  have  water  samples  collected  from  beaches  while  the 
survey  was  in  the  field,  and  I  also  consulted  contemporaneous  Department  of  Health  records  of 


'^  To  the  extent  that  trout  is  not  a  target  species  at  some  RUM  sites,  trout  biomass  would  be  the  wrong  type 
of  biomass  to  use  as  a  quality  indicator. 

'^  But  it  should  not  have  been  unexpected.  The  1989  version  of  MRIS  which  they  used  bore  the  disclaimer  that 
"any  data  retrieval  from  MRIS  is  not  intended  as  a  final  statement  on  river  related  resources  and  should  not  be  used 
as  a  substitute  for  on-site  field  surveys  needed  for  resource  evaluation." 
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water  quality  at  the  sites.  In  my  Southcentral  Alaslca  study,  I  used  contemporary  reports  of 
fishing  conditions  and  catch  to  track  the  quality  variable  during  the  course  of  the  survey.  Using 
a  variable  measured  at  some  unknown  time  in  the  past  is  asking  for  trouble. 

How  did  the  TER  research  team  react  to  the  news  that  the  variable  they  were  depending 
on  as  their  key  measure  of  site  quality  was  so  inaccurate  and  out-dated?  Did  they  drop  this 
variable?  Did  they  consult  the  new  MRIS  database  in  order  to  make  use  of  the  new  information 
on  biomass?  As  far  as  I  can  tell,  the  answer  to  both  questions  is  "no."  Neither  their  Report  nor 
any  of  their  internal  documents  available  to  me  suggests  that  they  either  inspected  the  new 
database  or  gave  any  consideration  to  modifying  their  biomass  variable.  Instead,  they  embarked 
on  a  Monte  Carlo  simulation  study  of  some  hypothetical  level  of  measurement  error  which,  they 
purported,  placed  an  upper  bound  on  the  degree  of  bias  in  coefficient  estimates  and  estimates  of 
consumer's  surplus.  In  my  view,  this  was  a  serious  error  in  judgment.  Using  a  variable  known 
to  be  so  problematic  without  checking  to  see  what  light  the  new  database  might  shed  on  these 
problems  is  irresponsible. 

I  have  examined  both  the  old  and  the  new  versions  of  MRIS  and  compared  the  trout 
biomass  measurement  used  in  the  TER  Report  against  the  data  in  the  new  version  of  MRIS.  The 
results  are  shown  in  Table  8.  Based  on  my  analysis,  I  have  reached  the  following  conclusions: 

(1)  Although  there  are  some  new  (post- 1989)  data  on  trout  biomass  in  version  3  of  MRIS,  most 
of  the  data  are  not  new,  as  can  be  seen  from  Table  8,  and  were  already  used  in  the  1989  version. 

(2)  Even  in  the  new  version  of  MRIS,  most  of  the  biomass  data  are  extremely  old.  Of  the  sites 
with  data,  9  sites  have  no  data  after  1976,  and  another  13  have  no  data  after  1984.  At  many  other 
sites,  even  though  there  are  some  later  data,  the  1976  data  constitute  a  significant  fraction  of  the 
available  information. 

(3)  In  most  cases,  the  data  are  very  sparse.  Often,  there  is  a  single  data  point  for  most  of  a  river  - 
-  sometimes  for  an  entire  river  —  and  this  data  is  imputed  to  every  other  segment  of  the  river  in 
order  to  fill  out  the  database.  On  the  main  stem  of  the  Kootenai  River,  for  example,  there  is  a 
single  piece  of  data  on  rainbow  trout  biomass,  dated  3/1/81,  and  this  same  value  is  listed  with 
monotonous  regularity  for  all  15  reaches  of  the  river."*  Similarly  with  12  of  the  15  other  RUM 
sites  listed  on  the  first  page  of  Table  8  —  there  are  only  one  or  two  pieces  of  data  on  biomass 
and  these  are  imputed  to  all  the  other  parts  of  the  site.  In  other  cases,  the  biomass  is  simply  listed 
as  zero.  In  these  cases,  zero  is  not  a  measure  of  biomass  but,  rather,  an  indication  of  "no  data." 
This  happens  with  14  of  the  RUM  sites. 

(4)  In  most  of  RUM  sites  where  the  data  are  sparse,  they  are  also  of  low  reliability.  MRIS  uses 
a  scale  of  1  -  9  to  rate  the  quality  of  the  biomass  data,  where  a  rating  of  1  -  3  denotes  data  based 
on  judgment  (i.e.,  a  guess);  a  rating  of  4  -  6  denotes  data  based  on  limited  measurements;  and 


I  skip  these  repetitions  in  Table  8. 
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TABLE  8:  ESTIMATES  OF  BIOMASS  (POUNDS  PER  1,000  FOOT  OF  RIVER) 


BIOMASS 
INTER 
RUM  SITE  REPORT 

Kootenai  River  86 

Yaak  River  9 

Upper  Rathead  River  71 

Stillwater  &  Whrteflsh  Rivers  1 3 

Rathead  River  157 

Middle  Fork  Rathead  River  78 

St.  Mary  River  etc  24 

Lower  Clark  Fork  57 

St.  Regis  River  47 

Rathead  River  70 

Middle  Clark  Fork  River  57 

Swan  River  90 

South  Fork  Rathead  River  9 

Teton  River  16 

Sun  River  29 

Dearborn  River  1 9 

Lower  Missouri  River  235 

Smith  River  63 

Belt  Creek  14 

Lower  Bitterroot  River  123 

Rock  Creek  167 

Blackfoot  River  70 


TROUT  BIOMASS  FROM  NEW  MRIS  DATABASE  (VERSION  3,  1994) 


10.8  (1986);  81.6  (1981) 
NO  DATA 

77(1985) 

7.3(1976) 

154(1985); 

77(1985) 

NO  DATA 
NO  DATA 


87.8(1976) 


NO  DATA 


440  (1976);  26.4,  4.4,  2.2  (1985);  5.3,  1.5,  39.6,  132, 121,  143,10.6,  9.7,  1 14.4  (nd) 
52.8,2.4,  39.6,  5.1, 1315.6,  4.6,  1227.6  (nd);  10.8  (1986) 

NO  DATA 

NO  DATA 

5.7,22(1976) 

5.7(1976) 

22,2.2(1976) 

82.4  (1969);15.4, 19.1  (1980);  502,  93.9,  (1983);602,  98.8  (1989);  132.4  (1976); 
355.7,  44.4,  320.1,  40  (1983);  46  (1989) 

22,110.7(1976) 

22,  (1976) 

132.4  (1976);  63.4, 25.5  (1984);  29  (1975) 

107.8  (1979);  13.2,  39.6  (nd) 

103.4,  11  (1982);  26.4,  66  (1976) 

4.2,  6.2  (1991);  9.2,  7.9  (1989);  14.7,  3.5  (1990);  20.5  (1982);  4.4  (1985) 


TABLE  8  continued 


RUM  SITE 


BIOMASS 

INTER 

REPORT 


TROUT  BIONMSS  FROM  NEW  MRIS  DATABASE  (VERSION  3,  1994) 


Clai1<  Fork  River  (Garrison-M 


Claik  Fork  River  (Warm  Spri 


30 


94 


Rini  Creek 

67 

Warm  Springs  Creek 

30 

Littie  Blackfoot  River 

50 

Tenmile  &  Prickly  Pear  Cree 

50 

Boulder  River 

23 

Judith  River 

6 

Musselshell  River 

60 

Upper  Bitterroot  River 

73 

Upper  Big  Hole  River 

56 

Lower  Big  Hole  River 

137 

Beaverhead  River 

260 

Ruby  River 

53 

Jefferson  River 

101 

Macfison  River 

316 

24.2,  2.2,  30.8  (1985);  32.8  (1989);  18.7  (1990);  25.7  (1991);  4.4,  8.8  (1985) 

8.6,  8.1,  1.1  (1989);  5.5,  4,  25.5  (1990);  43.6  (1989);  22.7,  60.7  (1990);  51.9  (1991) 

20.5  (1982);  4.4,  37.8  (1985);  52.8  (1989);  21.3,  81.4  (1990);  57,  77,  34.1  (1991) 

27.3  (1984);  67.1,  33.9  (1989);  36.3,  43.8  (1990);  37.2  (1991); 
68,  75.5  (1989);  47.3,  78.1,  70.6  (1990) 

68.4  (1991);  78.8  (1992);  72.6  58.5  (1989);  36.5,  73.7,  53.7  (1990);  84.3  (1991) 
49.9  (1992);  216.7  (1984);  149.8,  57.4,  106.3,  24.6,  152  (1989); 

207,  326.7, 134  (1990);  58.3,  3.1,  137.9,  51.7  (1991);  207, 120.6  (1992) 

72.6  (1976);  50.2  (1983) 

113.1, 152.2, 187.7  (1989);  60.9  (1990);  48.8,  45.5,  38.9  (1991) 

84.7  (1983);  14.1,  30.6,  51,  35.6  (1990);  23.8,  54.1,  85.4,  48  (1991) 
72.6,2.9(1976) 

70  (1974);  16,  41,  (1976) 
NO  DATA 

43.8  (1976);  66,  36  (1985);  42  (1987);  22  (1988);  64.9  (1985) 
63.4,  25.5  (1984);  33  (1985);  4,  8  (1992);  43.8;  52.8, 13.2  (nd) 
132.4  (1976),  87.8  (nd);  3.7, 21.1  (1983);64  (1980) 

64  (1983),  167.6  (1984),  132.4  (1976);  130,  60  (1986);  10  (1981);  8.8  (1980) 

109.6,  220  (1976);  444.6,424.4  (1984) 

66.4  (1982);  25.7,  13.9  (1976);  83.2,  24,  15, 13(nd);  5.9  (1976) 
8, 13.9  (1975);  12  (1970);  3.1  (1976) 

10.8  (1986);  61.8  (1985);  1 14  (nd) 

42,  107,  24,  75  (1989);  48,  54  (1992);  23,  44  (1994) 

41,  259  (1968);  87,  267  (1969);  69,  275  (1970);  87,  234  (1971) 

56,  298  (1972);  62,  243  (1973);  122,  248  (1974);  1 1 1,  308  (1975) 

91, 144  (1976);  129,  202  (1977);  189,  286  (1978); 

286,  291  (1979);  356,  266  (1980);  358  (1981);  273  (1982);  217, 

311  (1983);  117,  229(1986);  127,219(1987);  184,365(1988);  167, 

467  (1989);  88,  360(1990) 


TABLE  8  continued 

BIOMASS 

INTER 

RUM  SITE 

REPORT 

Missouri  River 

243 

Upper  Madison  River 

364 

TROUT  BIOMASS  FROM  NEW  MRIS  DATABASE  (VERSION  3,  1994) 


136.4  (1979);  45.8  (1982);  396  (1976) 

93.7,  273.9  (1984);  1 1 ,  137  (1967);  25, 127  (1968);  33,  1 18  (1969);  54, 
194  (1970);  113,  29,  20,  60,  251  (1971);  52,  93,  6,  9, 129,  14,  40, 
227  (1972);  24,  130,  156,  67, 167  (1973);  89,  203,  43,  214  (1974);  121, 
(1975);  169,  288  (1976);  95,  262  (1977);  167,  262  (1978);  75,  336  (1979); 
80  (1980);  127,  303  (1981);  127,  303  (1982);  148,  285  (1983);  109, 
332  (1984);  140,  206  (1986);  110,  176  (1987);  110,  187  (1988);  142, 
214  (1989);  81,  306  (1990);  101, 245  (1991);  105, 269  (1992);  41, 
281  (1993);71,89(1982);217,  165(1983);199,  53(1975);117,  61  (1976); 
348, 141  (1977);  257,  178  (1978);  235, 129  (1979);  209,  89  (1980);  1959, 
112  (1981);  234,  205  (1982);  191, 126  (1983);  213, 145  (1984);  316,  186  (1985); 
227, 161  (1987);  261,  600  (1994);  233,  130  (1977);  304,  200  (1978);  200, 
164  (1979);  175,  91  (1980);  742,  139  (1981);  385,  275  (1982);  287,  241  (1983) 
349,  248  (1984);  307,  213  (1985);  233  (1986);  255,  21 1  (1987);  414, 
262  (1988);  338,  392  (1989);  275, 154  (1990);  122, 142  (1991);  139, 
145  (1992);  92,  211  (1993);  206.1,  143.7(1984);  129.8,  56.5,  112.9,  336  (1976); 
108  (1970);  72.6  (1976) 


East  &  Lower  Gallatin  Rivers 


100  53.5,  62.9  (1976);  81.6  34,5  (1984);  24.9,  419  (1980);  82.3  (1984) 

89.8,  175.8  (1976);  197  (1977) 


Upper  Gallatin  River 

167 

Upper  Yellowstone  River 

119 

Middle  Yellowstone  River 

110 

Boulder  River 

67 

Stillwater  River 

70 

Red  Rock  River 

69 

Grasshopper  &  Prairie  Creek 

15 

Little  Bitterroot  River 

22 

Lower  Yellowstone  River 

155 

256  (1984);  43.8  (1976);  536  (1984);18  (1975);  21.8  (1976);  7  (1979) 
30.6,  73.5,  26  (1 986);  20.9,  77, 1 2.3  (nd) 

83.2  (1983);  38.5,  39,  65  (1984);  34,  43,  2.3  (1986);  25.1,  58.5, 14.7  (nd) 
167.9,  59,  5.9  (nd) 

52.6(1983) 

99.2  (1982);  7.5  (1986);  4.4,  31.5  (1986);42  (1989);  73  (1976);  27,  43.6  (1981) 
36  (1985);  12.5  (1986);  41.7  (1989);  43.8  (1976);  16,  65  (1989);  62.5  (1986) 

NO  DATA 

6  (1980);  39  (1992),  33(1986) 

NO  DATA 

175.8  (1976);  39,  65  (1984);  34,  43  (1986);  83.2  (1983);  38.5,  39,  65  (1984) 
34,  43,  2.3  (1986);  45.8,  26.2  (1982);  175.8  (1976) 


Silver  Bow  Creek 


NO  DATA 


TABLE  8  continued 


BIOMASS 

INTER 

RUM  SUE 

REPORT 

Missouri  River  (Fort  Benton) 

111 

Bighorn  River 

362 

Clarks  Fork  Yellowstone  Riv 

47 

Yellowstone  River  (Billings) 

40 

Bearpaw  Mountains 

28 

Missouri  River  (Ffve  Lakes) 

590 

Red  Lodge  &  Willow  Creeks 

6 

Ashley  Creek 

3 

TROUT  BIOMASS  FROM  NEW  MRIS  DATABASE  (VERSION  3,  1994) 

NO  DATA 
106  (1984);  372.5  (1985):1818  (1987); 

NO  DATA 
29,17,63,30,41  (1988); 
29.9,  21.3,  6.2,  29.3,  43.8  (1980) 
363.4,  216.5  (1983);  264  (1976) 

NO  DATA 

NO  DATA 


a  rating  of  6  -  9  denotes  data  based  on  extensive  measurements.  At  the  sites  with  sparse  biomass 
data,  the  quality  rating  is  generally  3  or  lower. 

(5)  Even  where  the  biomass  data  is  sparse,  it  can  also  be  highly  variable.  The  biomass  value  in 
one  part  of  a  river  can  be  substantially  different  from  that  in  another;  or,  the  biomass  value  in 
the  tributaries  can  be  substantially  different  from  that  in  the  mainstem.  At  the  Kootenai  River 
RUM  site,  for  example,  there  is  a  single  biomass  value  of  10.8  in  the  tributaries  and  a  single 
value  of  81.6  in  the  mainstem.  Similarly,  with  the  Teton  River,  the  Dearborn  River,  the  Smith 
River,  the  Lower  Bitterroot  River,  and  many  other  RUM  sites.  In  this  situation,  using  a  weighted 
average  of  the  two  biomass  values  (computed  to  two  decimal  places)  is  virtually  meaningless. 
Indeed,  using  any  single  measure  of  biomass  without  trying  to  represent  the  heterogeneity  of 
conditions  at  the  site  does  not  adequately  capture  the  quality  of  the  fishing  that  anglers 
experience.  The  aggregation  of  disparate  areas  into  a  single  RUM  site  would  still  be  a  severe 
problem  even  if  the  data  were  not  of  such  low  quality  and  so  many  years  out  of  date. 

(6)  At  a  few  RUM  sites,  instead  of  sparse  data  on  biomass,  there  is  extensive  coverage  both 
spatially  and  temporally.  This  is  especially  true  of  the  Clark  Fork  and  Madison  Rivers.  However, 
these  rivers  illustrate  the  extreme  variability  that  one  finds  with  biomass  across  repeated 
measures.  Figure  2  provides  an  illustration  ~  it  is  a  page  taken  at  random  from  the  many  pages 
of  data  on  the  Madison  River  in  the  new  version  of  MRIS.  In  this  case,  the  units  are  pounds  per 
mile  of  river  rather  than  per  1,000  ft.  It  shows,  not  surprisingly,  that  biomass  values  can  be  very 
different  for  different  species  of  trout  —  in  this  case,  rainbow  trout  versus  brown  trout.  Even  for 
a  given  species  of  trout  at  a  given  location,  the  biomass  values  vary  greatly  not  only  from  year 
to  year  but  also  across  repeated  measurements  in  the  same  year.  With  brown  trout,  for  example, 
there  are  two  separate  readings  in  1971,  one  using  the  mark-recapture  method  on  8/17/71  and  the 
other  using  a  log-likelihood  estimate  on  9/22/71;  the  first  generates  a  biomass  estimate  of  599, 
while  the  second  generates  an  estimate  of  1326.6,  more  than  twice  as  large.  In  1972,  there  are 
two  separate  estimates  based  on  the  mark-recapture  method,  dated  4/26/72  and  9/5/72,  and  a  third 
estimate  using  the  log-likelihood  method  on  9/29/72.  These  generate  estimates,  respectively,  of 
492,  682,  and  1280,  getting  on  for  a  three-fold  variation.  The  various  estimates  of  rainbow  trout 
biomass  in  1971  and  1972  show  a  similar  variation:  154,  319,  273,  47  and  212.  At  these  sites, 
the  variation  in  biomass  values  across  measurement  techniques  and  species  of  trout,  and  over 
time  (both  different  seasons  within  the  year  and  from  one  year  to  another)  is  compounded  by 
immense  spatial  aggregation. 

(7)  In  addition  to  the  problems  with  sparse,  unreliable,  and  outdated  data,  there  are  several  sites 
where  the  TER  biomass  values  appear  inconsistent  with  the  data  in  the  new  version  of  MRIS. 
At  the  Missouri  River  (Five  Lakes)  RUM  site,  the  TER  biomass  value  of  590  is  about  twice  what 
the  MRIS  data  suggest;  at  the  St.  Regis  River  RUM  site,  it  is  about  half  what  the  MRIS  data 
suggest.  Other  sites  where  there  are  serious  inconsistencies  include  the  Sun  River,  Belt  Creek, 
and  Warm  Springs  Creek.  In  addition,  there  are  the  sites  where  the  new  version  of  MRIS  offers 
no  data  on  biomass,  yet  there  are  biomass  values  in  the  TER  Report. 

(8)  Even  if  one  had  a  reliable  and  contemporaneous  measure  of  trout  biomass  at  a  site,  and  even 
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FIGURE  2:  SAMPLE  SCREEN  FROM  MRIS  VERSION  3 
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if  the  site  were  sufficiently  homogeneous  that  a  single  value  could  meaningfully  convey 
conditions  throughout  the  site,  it  is  not  clear  to  me  that  biomass  is  a  useful  way  to  capture  the 
quality  of  fishing  for  an  angler.  One  reason  has  already  been  alluded  to:  because  of  differences 
in  fish  size  and  other  characteristics,  biomass  values  are  not  necessarily  comparable  across 
species.  Even  for  a  single  species  of  fish,  biomass  may  be  an  imperfect  and  unreliable  measure 
of  the  quality  of  a  site.  It  may  be  a  good  indicator  at  the  extremes  —  if  there  is  no  biomass 
fishing,  will  be  bad;  if  the  biomass  is  huge,  fishing  cannot  fail  to  be  good.  But,  for  intermediate 
conditions,  it  may  not  be  a  good  indicator  of  fishing  quality;  it  may  not  even  be  positively 
correlated  with  fishing  quality.  At  any  rate,  this  correlation  needs  to  be  demonstrated  before  one 
can  have  great  faith  in  trout  biomass  as  the  sole  measure  of  fishing  quality  at  the  RUM  sites. 

Given  these  conclusions,  what  are  the  implications  for  the  estimation  of  the  RUM  model 
and  the  calculation  of  lost  consumer's  surplus  in  the  Upper  Clark  Fork  River  Basin?  In  my  view, 
there  are  two  distinct  problems  with  the  TER's  biomass  variable.  First,  as  I  have  just  indicated, 
even  if  biomass  were  measured  well,  it  may  not  capture  all  the  aspects  of  a  site  that  render  it 
desirable  for  fishing,  or  all  the  aspects  of  a  site  that  were  harmed  by  the  release  of  hazardous 
substances  in  the  Upper  Clark  Fork  River  Basin.  Second,  TER's  biomass  variable  is  not  measured 
well,  and  the  measurement  error  introduces  bias  into  the  RUM  model.  In  the  following  section, 
I  examine  the  bias  from  measurement  error.  In  section  4.6,  I  consider  the  possibility  that  there 
is  another  variable  that  captures  some  of  the  effects  of  the  release  of  hazardous  substances  in  the 
Upper  Clark  Fork  River  Basin. 


4.5  Monte  Carlo  Simulation  of  Measurement  Error 

Rather  than  trying  to  improve  their  biomass  measure  or  to  find  a  better  site  quality 
variable,  the  TER  research  team  responded  to  the  information  that  it  was  based  on  unreliable  and 
out-dated  information  by  conducting  a  Monte  Carlo  simulation  experiment  intended  to  bound  the 
effects  of  the  measurement  error.  From  the  simulation  experiment  they  concluded  that,  with  100% 
measurement  error  in  a  variable  in  a  nested  logit  model,  the  estimated  coefficient  on  the  variable 
is  attenuated  by  only  28.1%  below  the  true  value.  By  regressing  their  biomass  variable  on  some 
stock  data,  they  concluded  that  the  error  in  the  MRIS  measure  of  biomass  corresponded  to  a 
noise  level  of  70%  (i.e.,  the  error  in  the  variable  had  a  mean  of  zero  and  a  standard  deviation 
equal  to  0.7  of  true  value  of  the  variable).  They  concluded  that  this  implied  the  estimated 
coefficient  of  biomass  in  their  RUM  model  was  biased  downwards  by  19%.  Therefore,  they 
raised  the  fitted  coefficient  by  this  percentage  when  using  the  fitted  RUM  model  to  calculate  the 
loss  of  consumer's  surplus  from  pollution  in  the  Upper  Clark  Fork  River  Basin. 

Each  link  in  this  chain  of  reasoning  is  highly  questionable.  If  the  TER  research  team  had 
simply  bothered  to  inspect  the  new  MRIS  database  they  would  have  discovered  that  the 
measurement  error  in  the  biomass  variable  could  be  larger  than  70%.  And,  if  they  had  bothered 
to  review  the  extensive  statistical  literature  on  measurement  error  in  logit  and  related  nonlinear 
models,  they  would  have  discovered  that  the  bias  in  the  estimated  coefficient  could  be  many 
times  larger  than  28%.  Indeed,  they  would  have  seen  this  if  they  had  merely  looked  at  their  own 

31 


results  when  estimating  different  versions  of  the  RUM  model. 

A  distinctive  feature  of  the  entire  section  on  measurement  error  in  the  TER  Report  is  the 
lack  of  detail,  both  in  describing  the  experiment  and  in  relating  it  to  the  existing  literature.  While 
the  Report  provides  the  results  of  the  experiment,  it  omits  key  details  of  the  experiment's  design, 
including  the  number  of  observations  in  the  sample,  the  true  parameters  of  the  model  and  the 
assumed  distributions  of  the  variables.  Some  of  these  details  are  mentioned  in  an  internal  TER 
document  obtained  through  the  discovery  process,  entitled  "Effects  of  Data  Conditioning,  Sample 
Design,  and  Aggregation  on  Random  Utility  Model  Estimates:  Some  Monte  Carlo  Results"  by 
Steven  M.  Waters,  F.  Reed  Johnson  and  Robert  B.  Fowler.  However,  even  this  document  omits 
some  important  details.  It  is  not  clear,  for  example,  whether  the  RUM  models  estimated  in  the 
Monte  Carlo  experiment  fit  the  simulated  data  as  well  (or  poorly)  as  the  RUM  model  in  the  TER 
Report  fits  the  data  from  the  MOR  survey.  Nor  is  it  clear  whether  the  distribution  of  sites  shares 
in  the  Monte  Carlo  simulation  is  the  same  as  that  in  the  MOR  survey.'''  One  needs  some 
assurance  that  the  model  used  in  a  Monte  Carlo  simulation  matches  the  essential  features  of  the 
model  estimated  in  the  TER  Report  before  one  can  place  much  weight  on  its  results. 

One  also  wants  to  be  sure  that  the  results  of  the  Monte  Carlo  simulation  are  consistent 
with  the  existing  literature  on  measurement  error.  The  Report  makes  no  reference  to  the  literature, 
and  there  is  no  evidence  from  the  internal  documentation  that  the  TER  research  team  was  aware 
of  it  when  designing  the  Monte  Carlo  experiment.  This  is  a  serious  oversight,  since  the  literature 
is  extensive.'^  It  reaches  conclusions  substantially  different  from  those  in  the  TER  Report.  It 
shows  that  measurement  error  can  "wreak  havoc"  in  model  estimation  (Carroll,  1989).  Carroll, 
Ruppert  and  Stefanski  (1995)  provide  a  variety  of  empirical  examples  where  measurement  error 
in  logit  models  attenuates  coefficients  far  more  seriously  than  indicated  in  the  TER  Report. 

In  linear  models,  where  measurement  error  was  studied  extensively  in  the  1960s  and 
1970s,  there  are  some  standard  results  about  the  effects  of  measurement  error  on  regression 
coefficients  estimated  by  ordinary  least  squares.  The  simplest  case  is  where  there  is  measurement 
error  in  one  of  the  variables,  say  X,  and  the  error  is  additive  and  uncorrelated  with  either  the 
other  variables  in  the  model  or  the  error  in  the  regression  equation.  Thus,  instead  of  X  one 
observes  W  =  X  +  e.  When  one  regresses  the  dependent  variable  on  W,  the  result  is  a  consistent 
estimate  not  of  p  (the  true  regression  coefficient  for  X)  but  rather  of  yP,  where  y  is  given  by 

(1)  y  ^  cyi/ol  =  ai/iai  +  al)  <   I, 

where  o^  is  the  variance  of  the  true  variable  X,  cl  is  the  variance  of  the  observed  variable  W, 
and  al  is  the  variance  of  the  measurement  error.  The  estimated  coefficient  is  attenuated  by  the 


I  have  not  had  access  to  the  computer  code  and  raw  data  from  the  Monte  Carlo  study,  and  therefore  I  am 
not  in  a  position  either  to  resolve  these  questions  or  to  verify  the  accuracy  of  the  results  in  the  TER  Report. 

A  recent  book  on  the  subject  lists  over  300  items  in  statistics,  biometrics  and  econometrics,  most  of  them 
written  within  the  last  fifteen  years  (Carroll,  Ruppert  and  Stefanski,  1995). 
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factor  Y-  Fuller  (1987)  calls  ythe  reliability  ratio.  One  can  also  think  of  (crjol)  as  the  noise-to- 
signal  ratio. 

An  important  finding  of  research  in  the  last  ten  years  is  that  this  results  holds  up  rather 
well  in  many  nonlinear  models,  including  logit  and  related  models  estimated  by  maximum 
likelihood  or  other  methods.  This  fact  has  been  exploited  by  various  researchers  in  order  to 
develop  corrections  for  the  effects  of  measurement  error  (Carrol,  Ruppert  and  Stefanski,  1995). 
These  researchers  often  use  Monte  Carlo  simulation  experiments  to  test  the  efficacy  of  their 
proposed  correction.  I  have  examined  these  Monte  Carlo  simulations  and  found  that  using  y  as 
an  attenuation  factor  gives  a  good  prediction  of  the  effects  of  uncorrected  measurement  error  in 
logit  models.  This  holds  for  the  Monte  Carlo  simulations  in  Stefanski  (1985),  who  uses  300  and 
600  replications  per  experiment;  Schafer  (1987),  who  uses  100  and  300  replications  per 
experiment;  Whitemore  and  Keller  (1988),  who  use  50,  100,  and  500  rephcations  per  experiment; 
and  Rosner,  Willett  and  Spiegelman  (1989),  who  use  1,000  replications  per  experiment.  In  each 
case,  the  researcher  constructs  the  Monte  Carlo  experiment  by  first  picking  a  distribution  for  the 
true  value  X  with  some  given  mean  and  variance,  and  then  picking  a  distribution  for  e  with  a 
mean  of  zero  and  a  variance  a;  that  is  some  specified  proportion  of  a;.  For  example,  in 
successive  experiments  they  may  set  a^  =  O.la^,  1.05a;,  1.5a;,  and  2a^,  which  generates 
attenuation  factors  of  y  =  0.909,  0.5,  0.4  and  0.333,  respectively.  1  found  that  these  factors 
generally  predict  the  Monte  Carlo  simulation  results  well,  but  they  tend  to  underestimate  the 
attenuation  from  measurement  error  when  the  disu-ibution  of  e  has  heavier  tails  than  a  normal 
distribution  (Stefanski,  1985;  Schafer,  1987). 

The  TER  research  team  was  clearly  unaware  of  this  literature  when  they  designed  their 
own  Monte  Carlo  experiment,  because  they  constructed  it  in  a  rather  artless  manner.  They  picked 
a  distribution  for  X  with  some  mean,  |j^,  and  variance  a;,  and  then  specified  a  distribution  for 
e  with  a  mean  of  zero  and  a  standard  deviation  a,  that  is  some  specified  proportion  of  |i,  rather 
than  a^.  For  the  disu-ibution  of  X,  they  used  a  uniform  distribution  with  a  lower  support  at  zero. 
Regardless  of  the  upper  support,  this  type  of  uniform  distribution  has  a  coefficient  of  variation 
of  aj[i^  =  3"°^  =  0.5774;  i.e.,  a;  =  |iJ/3.  For  the  distribution  of  e,  they  used  a  normal  with  a  mean 
of  zero  and  a  standard  deviation  of  C^  =  Q\i^  where  they  varied  0  over  a  range  from  0. 1  to  1 .0. 
This  corresponds  to  an  attenuation  factor  of  y  where 

y  =   (nJ/3)  /  [(n^/3)  +  (OV;)] 
(2) 

=  (1/3)/ [(1/3) +  e^]. 

When  9  =  1,  y  =  0.25.  When  6  =  0.7,  which  TER  research  team  considers  to  hold  for  their 
biomass  variable,  y  =  0.409  and  the  regression  coefficient  is  understated  by  about  59%. 

However,  the  Monte  Carlo  simulations  in  the  TER  Report  show  much  less  attenuation  -- 
they  imply  attenuation  factors  (y-values)  of  .719  when  6  =  1,  and  .81  when  6  =  0.7.  In  other 
words,  while  there  is  an  established  body  of  results  in  the  published  literature  supporting  the 
finding  that  a  measurement  error  in  a  logit  model  corresponding  to  0  =  0.7  would  bias  the 
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coefficient  of  the  affected  variable  by  597c,  the  TER  study  finds  this  bias  to  be  only  19%.  This 
is  a  300%  difference. 

What  can  account  for  such  inconsistent  results?  One  possibility  is  that  the  result  in  the 
TER  study  is  caused  by  a  relatively  small  sample  size  —  the  TER  simulation  uses  only  100 
replications  per  experiment,  significantly  less  than  the  studies  mentioned  above.  Another  possible 
explanation  is  that  there  might  be  some  confounding  in  the  data  —  if  there  were  some  unintended 
correlation  between  the  error  term,  e,  and  the  other  stocha.stic  components  of  the  model,  or 
between  the  variable  measured  with  error  and  the  other  covariates,  this  certainly  would  affect  the 
degree  of  attenuation.  Since  I  do  not  have  access  to  the  data  used  in  the  TER  Monte  Carlo  study, 
I  cannot  check  this.  There  is  also  the  chance  of  programming  error  —  it  is  my  observation  that 
the  computer  programming  by  TER  research  team  tends  to  be  rather  sloppy  and  sometimes 
careless;  in  one  case,  discussed  below,  we  have  found  a  significant  error.  Since  I  have  not  seen 
the  programs  for  the  Monte  Carlo  study,  I  cannot  rule  this  out. 

A  third  explanation  rests  on  a  modification  to  the  Monte  Carlo  experiment  which  is 
mentioned  in  a  footnote  in  the  TER  Report.  This  states:  "if  the  noise  added  to  a  site  characteristic 
produced  a  negative  number,  a  level  of  zero  was  substituted."  I  have  not  seen  this  done  in  any 
of  Monte  Carlo  studies  in  the  published  literature.  It  reduces  the  effective  variance  of  the  random 
error,  e,  below  what  is  nominally  specified.  The  reason  why  the  TER  research  team  took  this 
unusual  action  is  that  there  would  otherwise  have  been  a  negative  value  for  the  explanatory 
variable  in  the  logit  experiment.  They  wished  to  rule  this  out  because  they  wanted  the  variable 
to  stand  in  for  biomass  (although  it  does  not  have  a  distribution  in  the  experiment  that  looks 
anything  like  the  distribution  of  biomass  as  shown  in  Table  8).  But,  the  real  implication  to  be 
drawn  from  the  occurrence  of  negative  values  of  (X+e)  is  that  the  assumption  of  an  additive  error 
of  measurement  is  inappropriate.  This  is  certainly  what  the  data  in  Table  8  suggest  --  the 
measurement  error  is  more  likely  to  be  multiplicative.  The  reason  for  this  is  that  a  false  reading 
on  the  downside  shrinks  the  true  value  of  the  biomass,  but  with  a  bound  of  zero,  while  a  false 
reading  on  the  upside  raises  it  without  any  similar  bound.  Either  the  simulation  should  be 
conducted  with  a  multiplicative  error  or  else,  if  one  uses  an  additive  error,  the  negative  values 
should  be  left  in  the  data.  This  is  what  the  other  researchers  did.  What  the  TER  research  team 
did  reduces  the  effective  variance  of  the  measurement  error,  at  a  rate  that  increases  with  6,  which 
undermines  the  purpose  of  the  simulation  experiment. 

A  fourth  possible  explanation  concerns  the  type  of  logit  model  used  in  the  study.  The 
Monte  Carlo  simulations  of  measurement  error  published  in  the  literature  deal  with  a  non-nested 
logit  model  estimated  by  maximum  likelihood;  the  simulation  in  the  TER  Report  involves  a  two- 
level,  nested  logit  model  estimated  by  full  information  maximum  likelihood  (FIML).  The 
measurement  error  affects  only  a  variable  in  one  of  the  lower  branches  at  the  lower  level.  It  is 
possible  that  when  a  nested  model  is  estimated  by  FIML,  this  mitigates  the  effect  of  measurement 
error.  The  fact  that  the  rest  of  the  tree,  which  is  free  from  measurement  error,  is  estimated 
simultaneously  with  the  part  of  the  tree  which  has  the  measurement  error  may  dampen  the  effects 
of  the  error.  If  so,  this  would  be  an  interesting  finding.  But,  it  would  not  apply  to  the  estimates 
of  lost  consumer's  surplus  in  the  Upper  Clark  Fork  River  Basin,  since  those  are  based  on  a 
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nested  logit  model  in  the  TER  Report  which  is  estimated  with  sequential  maximum  likelihood 
(SML).  The  SML  estimation  of  the  part  of  the  tree  affected  by  measurement  error  --  i.e.  the 
choice  among  river  sites  based  involving  the  biomass  variable  ~  is  the  same  as  a  simple,  non- 
nested logit.  Therefore,  the  Monte  Carlo  results  in  the  existing  literature,  rather  than  those  in  the 
TER  simulation,  would  apply  in  this  case. 

In  short,  on  the  basis  of  my  analysis  in  Section  4.4,  I  believe  that  the  biomass  variable 
in  the  TER  RUM  model  is  infected  with  substantial  measurement  error.  Based  on  my  analysis 
in  this  section,  I  beheve  that  this  measurement  error  can  have  a  substantial  impact  on  the 
coefficient  of  biomass  which  is  used  in  the  TER  Report  to  calculate  lost  consumer's  surplus 
resulting  from  the  release  of  hazardous  substances  in  the  Upper  Clark  Fork  Basin.  If  the 
measurement  error  corresponds  to  a  value  of  6  =  0.7,  as  the  TER  Report  maintains,  following 
the  published  literamre  I  believe  that  this  would  attenuate  the  coefficient  of  biomass  in  SML 
estimation  by  about  59%. 

Two  other  points  should  be  noted.  First,  there  may  be  corroboration  of  the  impact  of 
measurement  error  from  the  so-called  "elemental  site  model"  in  the  TER  Report,  in  which  fishing 
sites  are  defined  at  a  more  disaggregated  level  than  the  RUM  sites  considered  so  far.  Instead  of 
59  RUM  river  sites  and  38  RUM  lake  sites,  there  are  216  elemental  river  sites  and  93  elemental 
lake  sites.  The  difference  is  that,  whereas  the  RUM  sites  correspond  to  several  reaches  of  a  river 
or  set  of  rivers,  the  elemental  sites  correspond  to  individual  reaches  of  a  river.  The  TER  Report 
presents  a  model  of  fishing  participation  and  site/duration  choice  for  these  sites,  involving  trout 
biomass  among  the  determinants  of  choice  for  river  sites.  The  biomass  measure  for  the  elemental 
river  sites  is  taken  from  the  1989  version  of  the  MRIS  database.  Instead  of  having  to  average 
biomass  values  over  different  portions  of  the  RUM  site,  the  TER  research  team  uses  the  specific 
biomass  value  in  MRIS  for  the  reach  which  constitutes  the  elemental  site. 

I  have  not  had  the  opportunity  to  see  either  the  data  or  the  computer  code  used  in  the 
elemental  model.  However,  I  would  imagine  that  the  measurement  error  in  the  biomass  values 
for  elemental  sites  is  more  severe  than  for  RUM  sites.  Normally,  aggregation  of  heterogeneous 
sites  is  undesirable  -  indeed,  this  is  a  criticism  of  the  RUM  sites.  However,  to  the  extent  that 
measurement  errors  in  biomass  cancel  out  when  one  aggregates  individual  sites,  aggregation 
could  reduce  error  in  a  variable  measured  so  poorly  as  biomass.  Therefore,  I  would  expect  the 
attenuation  of  the  coefficient  on  biomass  to  become  far  worse  in  the  elemental  model  than  the 
RUM  model.  This  is  consistent  with  what  happens  -  the  coefficient  on  biomass  falls  by  a  factor 
of  138,  from  0.1681  in  the  RUM  model  to  0.0012165  in  the  elemental  model.  The  TER  Report 
neither  comments  on  this  nor  offers  an  explanation.  My  guess  is  that  it  is  a  consequence  of  a 
drastic  dechne  in  the  reliability  ratio  for  the  biomass  variable. 

Second,  measurement  error  in  one  variable  invariably  affects  the  coefficients  of  other 
variables  even  if  they  are  measured  without  error  and  are  statistically  independent  of  the  variable 
with  the  measurement  error.  This  arises  because  of  what  might  be  called  the  "perfect  prediction" 
property.  In  the  case  of  a  linear  model,  by  construction  the  ordinar>'  least  squares  regression  line 
passes  through  the  point  of  means:  at  the  mean  value  of  the  explanatory  variable,  the  fitted 
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equation  must  correctly  predict  the  mean  value  of  the  dependent  variable,  regardless  of  any 
measurement  error  or  any  misspecification  in  the  model.  Thus,  at  the  means,  the  OLS  estimates 
perfectly  fit  the  data.  An  analogous  condition  holds  for  logit  models  with  constant  terms  or 
dummy  variables:  as  will  be  discussed  below,  by  construction  maximum  likelihood  estimates  of 
these  models  perfectly  predict  the  actual  shares  of  the  alternatives  in  the  data.  For  other  nonlinear 
models  like  the  nested  logit  model  estimated  by  FIML  in  TER's  Monte  Carlo  simulation,  similar 
conditions  hold  on  average  if  not  exactly.  By  construction  in  these  models,  on  average  the  fitted 
model  must  predict  the  actual  shares  regardless  of  measurement  or  specification  error. 
Consequently,  when  measurement  error  in  one  variable  throws  that  coefficient  out  of  kilter,  the 
coefficients  of  the  other  variables  have  to  adjust  in  an  offsetting  direction  in  order  for  the  model 
to  continue  on  average  to  predict  perfectly.  Thus,  the  coefficients  of  all  variables  can  be  affected 
by  measurement  error  in  just  one  variable.  However,  they  are  affected  differently.  The  coefficient 
on  the  variable  measured  with  error  is  attenuated  towards  zero  ~  i.e.,  it  falls  if  positive,  and  rises 
if  negative.  If  there  is  but  one  other  variable,  as  in  the  TER  Monte  Carlo  experiment,  its 
coefficient  must  move  in  the  opposite  direction  —  i.e.,  away  from  zero  —  in  order  to  satisfy  the 
perfect  prediction  property.  The  TER  research  team  observe  the  change  in  the  coefficient  on  cost 
in  their  Monte  Carlo  experiments,  but  appear  not  to  grasp  the  reason  for  its  direction. 

They  do  note  an  important  implication  for  the  measurement  of  consumer's  surplus.  Since 
the  consumer's  surplus  associated  with  a  change  in  biomass  depends  essentially  on  the  ratio  of 
the  coefficient  on  biomass  divided  by  the  coefficient  on  cost,  measurement  error  has  a  double 
impact:  if  lowers  the  numerator  and  may  also  raise  the  denominator.  Thus,  it  can  attenuate  the 
estimate  of  consumer's  surplus  disproportionally  more  than  it  attenuates  the  coefficient  on 
biomass.  For  example,  when  0  =  1,  the  TER  Report  (page  79)  notes  that  while  the  Monte  Carlo 
simulations  show  a  28.1%  decrease  in  the  coefficient  on  biomass,  they  show  a  45.8%  decrease 
in  consumer's  surplus  per  choice  occasion.  Here,  the  attenuation  of  consumer's  surplus  is  63% 
larger  than  the  attenuation  of  the  coefficient  on  biomass. 

It  is  all  the  more  surprising  that  TER  research  team  proceed  to  overlook  this  double  effect 
of  measurement  error  in  Section  8.3  of  the  Report,  where  they  calculate  the  loss  of  consumer's 
surplus  from  recreation  in  the  Upper  Clark  Fork  River  Basin.  In  that  section,  they  allow  for  the 
attenuation  in  the  coefficient  of  biomass  (inadequately,  as  I  have  argued  above),  but  they  ignore 
the  potential  impact  on  the  coefficient  of  travel  cost  which  would  generate  a  disproportionally 
larger  attenuation  of  consumer'  surplus.  I  suppose  they  hoped  that  nobody  would  remember  what 
they  had  written  on  page  79. 


4.6  Other  Site  Characteristics 

I  noted  earlier  that,  in  addition  to  biomass,  the  RUM  choice  model  for  river  sites  includes 
a  variable  AESTHETICS.  This  is  a  dummy  variable  which  is  coded  "1"  or  "0"  (the  values  for 
the  river  sites  are  shown  in  the  first  column  of  numbers  in  Table  9).  The  Report  describes  it  as 
an  "aesthetics  rating  for  rivers"  and  says  that  it  comes  from  the  MRIS  database.  I  checked  the 
1989  MRIS  database  and  found  that,  in  the  Fishery  Assessment  section,  there  is  a  something 
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called  "esthetics  rating"  which  is  coded  as  "outstanding,"  "above  average,"  "average,"  "below 
average,"  "low,"  and  "national  renown."  These  codings  are  shown  in  the  second  column  of 
numbers  in  Table  9.  Since  RUM  sites  generally  consist  of  several  stream  sections,  sometimes 
quite  heterogeneous  in  quality,  there  can  be  several  different  esthetics  ratings  associated  with  a 
single  site.  In  those  cases,  I  have  listed  the  different  ratings  separately.'^  The  general  rule  seems 
to  be  that  the  TER  researchers  coded  a  site  as  1  in  their  AESTHETICS  variable  if  it  had  an 
MRIS  esthetics  rating  of  4,  and  0  otherwise.  However,  there  are  some  exceptions  to  this  rule, 
such  as  sites  52,  33,  35  and  41.  For  comparison,  I  have  included  the  TER  biomass  variable  in 
the  last  column  of  Table  9,  and  I  have  sorted  the  sites  by  the  MRIS  esthetics  rating.  While  all 
the  sites  with  the  highest  biomass  values  have  esthetics  ratings  of  4,  there  are  some  sites  with 
esthetics  ratings  of  4  that  have  distinctly  low  biomass  values  like  the  Kootenai  River  (site  1 )  and 
the  Blackfoot  River  (site  22).  The  biomass  value  for  Dearborn  River  (site  16)  also  seems  very 
anomalous.  The  juxtaposition  of  biomass  values  with  the  other  ratings  in  the  table  drives  home 
the  peculiarity  of  the  biomass  variable. 

I  wondered  when  I  saw  the  categories  for  the  esthetics  rating  because,  in  my  experience 
of  studying  fishing,  the  term  "river  of  national  renown"  is  used  synonymously  with  terms  like 
"blue-ribbon  trout  stream"  or  "trophy  trout  stream"  to  indicate  excellence  of  fishing,  not  scenery. 
To  investigate  the  matter  further,  I  consulted  a  trout  fishing  book.  Trout  by  Dick  Sternberg,  a 
volume  in  the  Hunting  &  Fishing  Library  published  by  Cy  DeCosse  Incorporated  in  1988.  This 
book  lists  blue-ribbon  trout  streams  across  the  country.  The  streams  in  Montana  that  it  lists 
coincide  pretty  well  with  the  streams  that  are  rated  "national  renown"  in  the  MRIS  database  and 
that  have  a  TER  AESTHETICS  value  of  1  (see  Table  9). 

To  resolve  the  matter,  I  contacted  George  Holton  in  Helena,  who  is  a  retired  Assistant 
Division  Administrator  in  Montana  DFWP.  He  was  the  Manager  of  the  Interagency  Stream  and 
Fisheries  Database,  of  which  MRIS  is  part.  He  was  kind  enough  to  send  me  a  document  entitled 
"Method  for  Assessing  the  Significance  of  River  Segments  and  Systems  for  Fisheries  Resources 
in  Montana,"  dated  November  11,  1985,  which  contains  the  instructions  for  coding  the  Esthetics 
rating  and  other  components  of  the  Fishery  Assessment  Section  of  the  MRIS  database.  These 
instructions  are  reproduced  in  Figure  3.  There  are  instructions  for  both  a  letter  rating,  A  -  F,  and 
a  numerical  code,  0  -  4,  where  0  is  low  and  4  high.  Footnote  10  refers  to  an  explanation  of  the 
esthetics  ratings  in  Attachment.  This  explanation  is  reproduced  in  the  lower  half  of  the  figure. 
It  is  evident  from  this  that,  while  the  middle  ratings  (A,  B,  C;  3,  2,  1)  of  esthetics  measure  scenic 
beauty,  the  highest  rating  (F;  4)  and  the  lowest  rating  (D,  E;  0)  measure  factors  associated  with 
trout  productivity  and  water  quality  —  factors  directly  influenced  by  releases  of  hazardous 
substances  in  the  Upper  Clark  Fork  River  Basin.  Since  the  TER  AESTHETICS  variable  picks 
out  the  sites  with  an  esthetics  rating  of  4,  it  serves  to  identify  the  most  productive  trout  streams 
in  the  state,  not  necessarily  the  most  scenic.  Moreover,  it  does  so  in  a  more  robust  manner  than 
the  biomass  variable. 


'^  In  the  MRIS  database,  there  is  no  esthetics  rating  for  Silver  Bow  Creek;  I  assume  it  was  off  the  scale  at  the 
low  end.  For  the  sake  of  completeness,  I  have  inserted  a  rating  of  0  in  the  table. 
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TABLE  9:  STTE  CHARACTtRISTTC  VARIABLES  FOR  RUM  SITES 


RUM 

TER 

MRIS 

BLUE-RIBBON 

MRIS 

TER 

SITE                          SITE  MAME  fTER) 

AESTHETICS 

ESTHETICS 

TROUT 

SCENIC 

BIOMASS 

(1) 

(2) 

(3) 

5  Rat^leadR^er 

1 

1.7 

157 

3  Upper  Rattiead  River 

1 

3 

70.94 

57  Missouri  Rrver  (Five  Lakes) 

1 

3 

590 

52  Ktssoun  River  (Fort  Benton) 

0 

2 

111.07 

38  Ma<4son  River 

1 

3.5 

315.95 

44  FkWde  Yelkjwstone  River 

1 

3 

110.48 

43  Upper  Yellowstone  Rrver 

1 

3.5 

118.95 

1  Kootenai  Rwer 

1 

4,  1 

4 

86.02 

21   Rock  Creek 

1 

4,  3 

4 

166.72 

39  Missouri  River 

1 

4,  2 

2 

243  43 

42  Upper  GallaDn  River 

1 

4,  2 

3.5 

167.13 

34  Lower  Big  Hole  River 

1 

4,  2 

3.5 

137.41 

53  Bighorn  River 

1 

4,  0 

3 

362.02 

17  Lower  Missouri  River 

1 

4,2,1,0 

2.7 

234.62 

22  Blackfoot  River 

1 

4,2,1 

2.5 

69.69 

33  Upper  Big  Hole  River 

0 

4,2.1 

2.5 

55.78 

13  South  Fort<  Flathead  River 

0 

4,3.1 

3 

9 

40  Upper  Macison  River 

1 

4,1,2 

4 

363.8 

41   East  &  Lower  GaJlalin  Rivers 

0 

4.  1 

2.5 

99.88 

35  Beaverhead  Rrver 

0 

4,1,0 

2.5 

260.24 

16  Deartxim  Rrver 

1 

3,  2 

0 

3 

18.7 

6  Middle  Fort<  Flathead  River 

0 

3,  2 

1 

4 

78 

18  Smith  Fliver 

0 

3,2.1.0 

0 

2 

63.37 

19  Beit  Creek 

0 

3,  1 

0 

2.2 

13.7 

45  Boulder  River 

0 

2 

0 

3 

67.06 

29  Boulder  River 

0 

2.  1 

0 

2 

22.8 

25  Flnt  Creek 

0 

2,  1 

0 

3 

67.24 

36  RubyRwer 

0 

2,  1 

0 

3.3 

52.82 

30  Ju(Sth  River 

0 

2,  1 

0 

2.3 

6.11 

46  Stillwater  River 

0 

2,  1 

0 

3 

69.59 

28  Tenmile  &  Prickly  Pear  Creeks 

0 

2,1,0 

0 

1.8 

49.85 

27  UtDe  Blackfoot  River 

0 

2,  0 

0 

2.8 

50.05 

24  Claik  Fori(  River  (Wami  Springs-Garrison) 

0 

1 

0 

2.5 

94.28 

37  Jeflerson  Fliver 

0 

1 

0 

3 

100.8 

14  Teton  River 

0 

1 

0 

2 

15.78 

32  Upper  Bittenrxjl  River 

0 

1 

0 

3 

73.32 

23  Clart<  Fori<  River  (Ganlson-Milltown) 

0 

1 

1 

3 

29.93 

31   Musselshell  River 

0 

1 

0 

2.5 

60.4 

20  Lower  BitteniDOt  River 

0 

1 

0 

3 

123,07 

2  Yaak  River 

0 

1 

0 

3 

8.66 

12  Swan  River 

0 

1 

1 

2 

90 

9  St  Regis  River 

0 

1 

0 

2.5 

46.61 

10  Flathead  River 

0 

1 

0 

2.5 

69,93 

11   Wddle  Clari<  Fori<  River 

0 

1 

0 

2.3 

56.77 

47   Red  Rock  River 

0 

1 

0 

2.5 

68.82 

54  Ctarte  Fork  Yeltowstone  River 

0 

1 

0 

2.2 

47.19 

50  Lower  Yeltowstone  River 

0 

1 

0 

2.5 

155.49 

48  Grasshopper  &  Prairie  Creeks 

0 

1,  0 

0 

2.9 

14,55 

15  Sun  River 

0 

1 

0 

0 

1 

28.68 

59  AshteyCreek 

0 

1 

0 

0 

1 

3.42 

58  Red  Lodge  &  Willow  Creeks 

0 

1 

0 

0 

1.3 

6 

8  Lower  Clari<Fort( 

0 

1 

0 

0 

1.5 

56.59 

4  Saiwater  &  Whitefish  Rivers 

0 

1 

0 

0 

1.75 

12.79 

7  Si  Maiy  River  etc 

0 

1 

0 

0 

2.5 

24.48 

55  Yellowstone  Rrver  (Billings) 

0 

1 

0 

0 

2.2 

39.56 

56  Bea^jaw  MomBlins 

0 

1 

0 

0 

2 

2829 

51   S»ver  Bow  Creek 

0 

0 

0 

1 

0 

26  Wami  Springs  Creek 

0 

0 

0 

2 

30 

49  Utile  Bitterroot  River 

0 

5 

0 

3 

22 

NOTES:    (1)4  =  national  renown:  3  =  outstandng:  2  =  above  average:  1  =  average,  0  =  below  average,  low. 

(2)  1  =  Isted  as  Wue-rttion  tnxit  stream  in  Hunfing  &  Fishing  Library  volume  on  Trout  0  =  not  so  listed. 

(3)  4  =  outstandng  (excellent):  3  =  substantial  (high):  2  =  moderate:  1  =  limited. 


FIGURE  3:  INSTRUCTIONS  FOR  ASSIGNING  AESTHETICS  GRADE  IN  MRIS  DATABASE 


Criterion  III.  Assignment  of  Esthetics  Grade 

Esthetics  rating  —                       Grade 

A  3 

B  2 

C  1 

D  0 

K  0 

F  U 


10/ 


.   See  explanation  of  esthetics  ratings  in  Attachment, 


ESTHETICS  RATINGS.  Esthetics  are  rated  A  (high)  through  E  (low).  Features  that 
detract  from  esthetics  include:  pollution,  dewatering,  channelization,  riprap 
(particularly  car  bodies  and  discarded  building  materials),  mine  tailings,  a 
busy  highway  along  stream  and  severe  land  abuse.   As  a  guide: 

A    A  stream  of  outstanding  natural  beauty  in  a  pristine  setting. 

B  A  stream  comparable  to  A  except  that  It  may  lack  pristine 
characteristics.  Presence  of  human  development  such  as  roads,  farms, 
etc.,  usually  comprise  the  difference  between  B  and  A. 

C  A  stream  with  natural  beauty  but  of  a  more  common  type  than  listed 
under  A  and  B.   A  clean  stream  in  an  attractive  setting. 

D    A  stream  and  area  with  fair  esthetics. 

E     A  stream  with  low  esthetics. 

F    A  stream  of  national  renown. 


If  one  wants  a  measure  of  scenic  quality,  this  is  provided  by  a  variable  in  the  Recreation 
Assessment  of  the  MRIS  database  called  "Scenic  Quality".  This  rates  reaches  of  rivers  using  5 
categories:  1  -  Exceptional;  2  -  High;  3  -  Moderate;  4  -  Limited;  and  0  -  Unknown.  Figure  4 
presents  descriptions  of  these  categories,  taken  from  Appendix  E  of  the  current  MRIS  manual, 
Montana  Rivers  Information  System:  Reporting  User's  Manual,  February  1 994.  The  scenic  quality 
ratings  for  the  RUM  sites  are  shown  in  the  penultimate  column  of  Table  9;  in  this  case  I  have 
taken  a  simple,  unweighted  average  of  the  ratings  for  the  individual  stream  segments  that  make 
up  a  RUM  site.  While  many  scenic  sites  also  rank  high  on  the  esthetics  ratings,  and  conversely 
for  unattractive  sites,  there  are  important  cases  of  divergence.  The  Little  Bitterroot  River  (site  49) 
has  a  high  rating  for  scenic  quality,  but  a  low  one  for  esthetics.  Conversely,  site  52  (Missouri 
River,  Fort  Benton)  has  a  high  rating  on  esthetics  but  a  relatively  low  one  for  scenic  quality. 
Similarly  with  respect  to  AESTHETICS.  This  reinforces  my  conclusion  that  the  AESTHETICS 
variable  in  the  TER  Report  is  measuring  trout  productivity  rather  than  scenic  beauty. 
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FIGURE  4:  CODING  OF  SCENIC  QUALITY  IN  MRIS  DATABASE 


LOOKUP  TABLE  VI  -  SCENIC  QUALITY 

1=  EXCEPTIONAL 

Outstanding  scenic  quality.  For  these  segments,  landforms,  vegetation  patterns,  and  water  features  combine  to 
create  unique,  highly  memorable,  and  harmonious  visual  settings.  Views  along  the  river  and  away  from  the  river 
to  surrounding  scenery  are  highly  diverse,  providing  river  users  with  scenery  that  is  spectacular  and/or  not  common 
on  other  rivers  in  the  region.  If  buildings,  roads,  and  other  cultural  modifications  are  present,  they  either  add 
favorably  to  or  do  not  intrude  on  visual  quality  for  river  users. 

2  =  HIGH 

High  scenic  quality.  For  these  segments,  landforms,  vegetation  patterns,  and  water  features  combine  to  create  a 
highly  memorable  and  visually  pleasing  setting,  although  one  that  may  be  more  common  to  the  region.  Views  along 
and  away  from  the  river  axe  highly  diverse  and  cultimJ  modifications,  if  present,  either  add  to  or  do  not  detract 
from  the  visual  setting. 

3=M0DERATE 

Moderate  scenic  quality.  For  these  segments,  landforms,  vegetation  patterns,  and  water  features  along  the  river 
combine  to  create  harmonious  but  common  visual  settings.  Views  along  and  away  from  tlie  river  are  somewhat 
varied,  but  lack  a  high  degree  of  contrast  and  diversity.  Encroachment  of  cultural  modifications  may  be  evident, 
and  either  adds  little  to  or  detracts  from  visual  quality. 

4=LIMITED 

Low  scenic  quality.  For  these  segments,  landforms,  vegetation  patterns,  and  water  features  combine  to  create  visual 
seUings  lacking  in  variety  and  contrast.  Views  along  and  away  from  the  river  are  monotonous  and  common. 
Cultural  modifications  may  dominate  and  detract  from  visual  quality. 

0=UNKNOWN 

Insufficient  information  available  to  evaluate  scenic  quality. 
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5.  The  RUM  Model:  Structure  and  Estimation 


5.1  Overview 


The  RUM  model  developed  by  the  TER  research  team  operates  on  a  weekly  time-step, 
with  three  nested  levels  of  angler  choice:  (1)  Whether  to  take  a  fishing  trip  this  week  and,  if  so, 
how  many  trips  (1  trip  vs.  more  than  1  trips)?  (2)  Given  that  the  angler  is  taking  a  trip  this  week, 
should  he  go  to  a  river  or  lake  site?  (3)  Given  the  type  of  site,  to  which  specific  river  or  lake  site 
should  he  go,  and  for  how  long  (1  day,  2  days,  3  days,  or  4+  days)? 

The  general  type  of  structure  model  is  not  unreasonable.  Indeed,  as  the  Report  notes,  it 
is  patterned  after  a  model  that  I  developed  with  Richard  Carson  for  sportfishing  in  Southcentral 
Alaska  in  1986,  and  a  related  model  that  Richard  and  I  developed  for  sportfishing  in  Southeastern 
Alaska  in  1988,  on  which  Steven  Waters  worked  as  our  research  assistant  and  which  served  as 
the  basis  for  his  Ph.D  dissertation  under  Richard  at  U.C.  San  Diego.  My  concerns  center  on  two 
issues:  the  limitations  of  the  data,  and  the  specific  details  of  the  tree  structure. 

With  regard  to  the  data  used  to  estimate  the  model,  I  have  identified  the  following 
limitations: 


RUM  DECISION  TREE 

DATA  LIMrrATIONS 

Choose  to  recreate? 
How  many  trips  to  take  this  week: 
0,  1  or  2+  trips? 

Over-estimation  of  anglers  in  the  MOR  telephone 
survey,  combined  with  severe  under-estimation  of 
the  number  of  fishing  trips  due  to  progressively 
increasing  nonreporting  in  the  MOR  panel  survey, 
make  it  impossible  to  obtain  a  reliable  estimate 
of  this  choice. 

River  or  lake  site  for  this  trip? 

Over-estimation  of  trips  in  the  MOR  survey  to 
river  sites  in  summer,  and  under-estimation  in 
winter,  make  this  unreliable. 

To  which  lake/river  site,  and  how 
long  a  trip  (1,  2,  3,  or  4+  days)? 

The  nonreporting  in  the  MOR  survey  gives  rise  to 
a  tiny  number  of  observations  relative  to  the  large 
number  of  sites.  Oversampling  of  long  trips  and  of 
avid  anglers,  and  inclusion  of  individuals  who  are 
outhers,  bias  results.  Errors  in  measuring  variables 
bias  coefficient  estimates. 
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I  observed  at  the  beginning  of  this  report  that  any  economic  model,  no  matter  how  excellent  in 
theory,  will  be  of  little  practical  value  if  it  is  applied  to  unreliable,  unrepresentative  or 
contaminated  data.  I  have  come  to  the  conclusion  that  the  TER  model  of  angler  behavior  falls 
into  this  category.  In  my  view,  the  data  from  the  MOR  survey  are  not  reliable  for  estimating  the 
top  two  tiers  of  the  model.  Those  two  tiers  should  be  dropped  from  the  model,  and  the  choices 
in  those  tiers  should  simply  be  benchmarked  against  the  far  more  extensive  and  reliable  data  from 
the  National  Survey  of  FHWAR  and  the  Statewide  Angling  Pressure  Survey.  For  reasons  spelled 
out  below,  I  am  not  certain  at  this  time  about  the  reliability  of  the  data  used  for  the  bottom  tier 
of  the  model.  I  am  sure,  though,  that  these  data  are  inconsistent  with  the  specific  structure  of  the 
TER  model. 

In  this  section  I  discuss  three  aspects  of  the  model:  the  assumptions  implicit  in  the  pooling 
of  data  used  to  estimate  the  model,  the  assumptions  implicit  in  the  model  structure,  and  the 
results  when  the  model  is  estimated. 


5.2  Pooling  Disparate  Data 

One  of  the  most  striking  features  of  TER' s  RUM  model  is  the  contrast  between  the  small 
number  of  respondents  in  the  MOR  survey  reporting  their  fishing  trips,  on  the  one  hand,  and  the 
relatively  large  number  of  sites  that  the  TER  research  team  chose  to  model  —  97  sites  and  only 
1736  trips,  producing  an  average  of  only  18  trips  per  site.  Moreover,  165  of  the  trips  come  from 
two  individuals  who  are  clearly  outliers. 

The  only  way  the  TER  research  team  could  model  choices  among  so  many  sites  with  so 
little  data  is  by  making  three  strong  assumptions  about  angler  preferences  and  behavior: 

(A)  Winter  fishing  and  summer  fishing  are  sufficiently  similar  activities  that  they  can  be  pooled 
and  modelled  with  a  single  set  of  demand  functions. 

(B)  The  fishing  preferences  and  behavior  of  anglers  in  the  different  parts  of  Montana  are 
sufficiently  similar  that  the  data  for  all  resident  anglers  can  be  pooled  and  modelled  with  a  single 
set  of  demand  functions. 

(C)  All  river  fishing  sites  in  the  TER  study  area  in  Montana,  and  all  lake  fishing  sites,  are 
sufficiently  similar,  and  the  explanatory  variables  included  in  the  RUM  model  are  so 
comprehensive  (and  measured  with  such  accuracy),  that  one  "generic"  set  of  demand  functions 
can  be  fitted  for  all  river  sites,  and  another  for  all  lake  sites.  The  demand  functions  for  all  rivers 
depend  on  exactly  the  same  explanatory  variables,  and  these  variables  have  exactly  the  same 
coefficients,  including  intercept  term,  for  all  rivers.  Similarly,  the  demand  functions  for  all  lakes 
depend  on  exactly  the  same  explanatory  variables,  and  these  variables  have  exactly  the  same 
coefficients. 

These  assumptions  greatly  simplified  the  task  of  the  TER  research  team.  They  permitted 
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it  to  pool  all  trips  in  a  single  estimation,  and  they  made  it  possible  to  estimate  demand  functions 
for  sites  where  there  were  few  observations  —  indeed,  for  five  RUM  sites,  where  there  were  no 
observations.  All  of  this  was  due  to  the  immense  power  of  assumptions.  Who  needs  data  when 
he  has  assumptions!  The  TER  research  team  made  no  attempt  to  investigate  whether  these 
assumptions  hold.  They  took  them  for  granted.  Based  on  my  own  experience  of  studying  fishing, 
I  doubt  that  the  assumptions  hold. 

With  regard  to  the  pooling  of  summer  and  winter  fishing  trips,  while  this  may  be  of  minor 
concern  in  regions  with  an  unchanging  climate  year-round,  such  is  not  the  case  in  Montana.  In 
northern  regions,  in  my  experience,  winter  and  summer  fishing  are  very  different  experiences, 
and  appeal  to  different  kinds  of  angler.  As  a  colleague  once  remarked  to  me  when  we  were 
studying  fishing  in  Alaska,  while  the  best  substitute  for  fishing  at  one  site  in  summer  is  fishing 
at  another  site,  the  best  substitute  for  fishing  at  one  site  in  winter  is  drinking  in  a  bar.  In  our 
study  of  Southcentral  Alaska  we  found  a  considerable  difference  in  patterns  of  angler  behavior 
and  site  preference  for  summer  and  winter  fishing.  This  does  not  mean  that  there  is  little  value 
from  fishing  in  winter,  or  that  the  consumer's  surplus  from  winter  fishing  is  lower  for  those  who 
participate  in  it  than  the  consumer's  surplus  from  summer  fishing.  It  is  just  that  these  are 
different  activities  and,  in  my  judgment,  they  should  be  modelled  separately. 

In  this  case,  however,  if  the  TER  research  team  had  separated  summer  and  winter  fishing 
trips,  it  would  have  been  left  with  only  1196  summer  trips  and  540  winter  trips  with  which  to 
account  for  97  sites  (Table  10).  The  sample  size  is  probably  too  small  to  permit  a  reliable 
analysis  of  winter  fishing  at  the  97  sites. 

The  data  in  Table  10  also  show  why  it  may  be  inappropriate  to  lump  anglers  from  all 
areas  into  a  single  set  of  demand  functions.  Not  only  should  the  out-of-state  angler  have  been 
dropped,  but  also  the  resident  anglers  from  the  North  Central  Area.  Their  lake/river  choice  is 
radically  different  from  the  rest  of  the  sample.  I  suspect  that  many  of  the  fishing  sites  they 
patronize  are  among  those  excluded  from  the  RUM  sites  ~  far  more  so  than  for  the  other  anglers 
in  the  sample. 

Table  1 1  shows  the  distribution  of  fishing  trips  by  anglers  in  each  area,  broken  down  by 
season.  The  difference  in  distribution  of  trips  among  individual  sites  provides  strong  support  for 
treating  winter  and  summer  fishing  separately.  The  table  also  casts  doubt  on  the  reasonableness 
of  lumping  all  anglers  into  a  single  set  of  demand  functions.  This  is  illustrated  in  Figure  5,  which 
shows  the  drainages  in  which  Billings  area  residents  took  fishing  trips,  whether  winter  or 
summer.  Not  surprisingly,  anglers  from  this  area  went  fishing  at  RUM  sites  in  the  same  general 
area,  RUM  sites  in  surrounding  areas,  and  RUM  sites  in  southwest  Montana.  They  did  not  go  to 
RUM  sites  in  drainages  5,  7,  8,  and  11.  Conversely,  Figure  6  shows  that  anglers  from  the 
Kalispell  area  did  go  to  those  sites,  but  did  not  go  to  RUM  sites  1,  9,  13,  17  or  20.  Is  it 
reasonable  to  assume  that  anglers  in  each  area  are  choosing  from  the  full  set  of  97  sites  when 
they  go  fishing?  If  so,  does  every  site  enter  every  angler's  utility  function  in  a  purely  generic 
manner?  The  TER  research  team  assumes  a  (convenient)  answer  to  those  questions,  without 
establishing  that  the  assumption  is  correct. 
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TABLE  10:  BREAKDOWN  OF  MOR  TRIPS  BY  RIVER  vs  UVKE  SITES,  BY  SEASON  &  AREA  OF  RESIDENCE 


SUMMER  TRIPS 


WINTER  TRIPS 


AREA  OF  RESIDENCE 


LAKE         RIVER     %  RIVER 


LAKE         RIVER      %  RIVER 


BILLINGS  AREA 
GREAT  FALLS  AREA 
NORTH  CENTRAL  AREA 
HELENA  AREA 
SOUTH  WEST  AREA 
MISSOULA  AREA 
KAUSPELAREA 

OUT  OF  STATE 

TOTAL 


45 

119 

72.6% 

80 

67 

45.6% 

15 

3 

16.7% 

98 

110 

52.9% 

123 

153 

55.4% 

86 

147 

63.1% 

75 

73 

49.3% 

0 

2 

100.0% 

522 

674 

56.4% 

15 

40 

72.7% 

21 

21 

50.0% 

7 

2 

22.2% 

86 

38 

30.6% 

84 

24 

22.2% 

34 

39 

53.4% 

57 

72 

55.8% 

304 


236 


43.7% 


T«aLE11  OtSmeLmCNOFTniPSMMOR  SURVEY.  BY  SEASON  t  AREA  OF  ORIGIN 


Rll  1  nOS  AREA 
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HELENA  AREA 

S  WESTERN  AREA 
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KALISPELAHEA 

OPHP 

HESCENTS 

RESIDENTS 

RESIDENTS 

RESIDENTS 

RESIDENTS 

RESIDENTS 

RESIDENTS 

DRAMAOERLM 
SITE 

SHE  NAKC  (TERJ 

WNTEB    SUHftCH 

WINTER    SUKWER 

WKTER    SL*».(EB 

WKTEH     SUI*IEB 

WINTER    SUMJIER 
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WKTER    SUMiei 

78 

ChHtaryyii— III* 

aa% 

OOX          OOX 

08% 

130% 

33% 

0/«K 

3e 

Ril^rAo 

0.5% 

» 

BaMtfiiainMr 

1«% 

72% 

48 

OnaXon^  t  Praia  Cnata 

47 

RalHadinMi 

22% 

33 

L«va  Bq  Hok  !«>« 

0B% 

33% 

21% 

34 

Lo<n>eigHotol«>« 

1<(K 

3.7% 

11.0% 

00% 

QTfc 

» 

Lcww  BaKTOOl  Rnw 

0.4% 

82% 

e»% 

73 

ByrunRMWor 

7.1%          34% 

04% 

87 

LjtoConv 

14% 

17% 

32 

U|«>9  aamtm  Rmi 

39% 

22 

BIkMooiRkv 

41% 

1B% 

0.4% 

27% 

107% 

0J% 

70 

U*CA« 

07% 

08%           10% 

0S% 

55% 

52% 

06% 

» 

9  RavalVv 

14% 

26% 

11 

ltfelJ»CWiF(>l>l««v 

82% 

183% 

e 

LpMrCMiForii 

os« 

3.9% 

07% 

«i 

BUUto 

2J% 

07% 

es 

CMind  Gaga  A  Nomn  Ranrvon 

IjK 

St 

Sln8a»CiM> 

21 

HocKCMik 

82% 

3  4% 

23 

avk 

10% 

04% 

178% 

30% 

26 

AamS^mfpOwk 

27 

LUsBlKMacxRfW 

07% 

63% 

04% 

17% 

74 
24 

GacraslownLBto 

0^           0S% 

27  6% 

210% 
0.7% 

20i% 

43% 

25 

Ffc<Cn* 

03% 

07% 

17% 

07% 

ae 

RvLito 

2.3% 

07% 

64 

KlVY  Hcna  Rmwvv 

04% 

1>l% 

83 

Sn<hLato 

es% 

m 

RM^a■dL•to 

14% 

1B%           1.0% 

0S% 

1.4% 

1»2% 

129% 

101% 

16:9% 

49   una  BtMnoa  Rm> 


13 
13 
13 


K   Taty  Sykn  t  Li«ra  Ulw 

86   HQland.  LnlMrg  &  Gl»3er  Laka* 

62  WrMWiLate 

63  Atfiley  1  ume  Bnaincl  Lalws 
4  Siawls  i  WhaeAsX  Ri.«n 

10   RtfwKjnw 
96   VxLataiaC 
65   SavLMa 

12  SnvnRaw 
SO  Ad^Craek 

6  MdJe  Fait  Ralhead  RM«r 

13  ScK«i  Fork  FtaihaKj  R^w 
3  Un3«r  FhiJVKj  Rnbt 

42    upper  Saiasi  Rmt 
38   MadKuRivo 
41    Ema  4  Lowar  Gafatii  Riwra 
96  DtfmeLato 
37  JrfiaraonRMr 
29   SnAtocRm 
60  Laka  Koocanas 
81    Thonvaon  Laitaa 
1 


08% 
7  8% 

16% 


26% 
0.9% 


as% 

(M% 


05% 

05% 

4.0% 

16% 

0.9% 

4J% 

0.8% 

1.0% 

3.7% 

ei% 

1S% 

11% 

32% 

yB% 

3.7% 

47% 

14% 

22% 

2  YaakFkiec 

40  Utw>tetoaiRk« 

80  Habgan  Rnancr 

78  EnraL^ia 

14  Talon  Rmt 
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7  StUnyRiwatc 
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30  JudttiRMT 

52  Unoui  nw  (Fon  Barton) 

19  Bat  Cm* 

18  Sni«iRKac 
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57  Usaoji  Rivar  (FWa  L^<as) 

17  Lowar  Ifcsxji  Rjvar 
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39  hhaouiRMT 
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26  Tarn*  4  Pnc*»»  Paa/ Oaalia 

76  Canyon  FarTy  Raaarww 
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66  Lnnr  SI  Uaiy  Laka 

15  SuiRmc 


0.6% 

0.6% 

95% 
48% 

14% 
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48% 

24% 

82% 

33.3% 

167% 
333% 

0.6% 

24% 

4  1% 
4  1% 
07% 
116% 

222% 

187% 

12% 

71% 
36  7% 

68% 
88% 

12% 

7.1% 
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07% 
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24% 

07% 
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333% 

1*lk 

1/4% 
05% 


1jO% 
05% 


142% 
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00% 
07% 
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2m 
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07% 
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0  8%         189% 
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04% 
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54  Ckrts  Fort  YafcwOora  RNar 
82  Coonay  Raaarioa 

46  n#l^■R^af 
84  Bi^loniLalia 

50  Lmai  y^DMlona  Rmt 

TOTAL 

NJIieB^  OF  TRPS 


06% 

24% 

20% 

37% 

30.0% 

15.9% 
3.0% 
12% 

3«% 

6i% 

07% 

3B% 
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73% 

3«% 

115% 
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1C00% 
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42 
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0.4% 
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04% 
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0.4% 

0«% 

05% 
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0.4% 

0.0% 

1.1% 
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04% 
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5.3  Model  structure 

An  unusual  feature  of  the  TER  model  is  that  it  combines  choice  among  sites  with  choice 
of  trip  length  in  one  nest  of  the  model.  This  implies  that  an  angler  sequentially  chooses  first 
whether  to  go  to  a  river  or  lake,  and  then  whether  to  take  a  long  or  short  trip.  However,  in  the 
MOR  data,  longer  multi-day  trips  are  more  frequently  associated  with  lake  sites  than  river  sites. 
Few  anglers  take  long  trips  to  river  sites;  they  take  these  trips  to  lakes.  One  reason  is  that  anglers 
have  cabins  at  lakes  more  often  than  at  river  sites.  In  my  view,  the  choice  of  trip  length  and  lake 
vs.  river  site  is  simultaneous  for  most  anglers,  or  sequential  in  the  reverse  order:  anglers  first 
decide  what  type  of  trip  length  (e.g.  going  on  a  week's  vacation  vs.  going  away  for  the  day  or 
overnight)  and  then  where  to  go.  Moreover,  they  are  likely  to  consider  different  sets  of  sites 
depending  on  the  length  of  trip  that  they  have  chosen  ~  many  places  near  home  would  make  no 
sense  as  a  destination  for  a  week's  vacation,  and  many  sites  very  distant  would  make  no  sense 
as  a  day  trip,  given  the  time  needed  to  get  there  and  back.  This  is  not  captured  in  the  TER 
nesting  structure. 

The  model  structure  also  pays  inadequate  attention  to  the  type  of  fishing  --  what  species 
of  fish,  and  what  method  of  fishing  (e.g.,  fly-fishing  vs.  bait-fishing).  This  is  introduced  into  the 
middle  nest  of  the  TER  model,  but  nowhere  else.  Thus,  if  the  target  species  is  not  trout,  or  if  the 
trip  is  not  fly-fishing,  in  the  TER  model  this  makes  the  angler  more  likely  to  choose  a  lake  site 
versus  a  river  site  in  the  lake/river  split.  But,  nothing  in  the  model  accounts  for  the  choice  of 
target  species  or  the  choice  of  fishing  mode.  Also,  the  model  does  not  allow  the  choice  of  target 
species  or  fishing  mode  to  influence  the  choice  of  a  particular  river  site/lake  site,  or  the  choice 
of  trip  length.  In  the  Southcentral  Alaska  study,  I  allowed  the  angler  to  choose  a  target  species 
and  this  influenced  the  choice  of  site  because  different  choice  sets  of  sites  were  associated  with 
different  species.  By  contrast,  the  TER  model  assumes  that  the  species  and  mode  of  fishing  have 
no  influence  on  which  river  site,  or  which  lake  site,  an  angler  chooses. 

Why  does  model  structure  matter?  What  difference  does  the  nesting  structure  make?  From 
a  statistical  point  of  view,  it  makes  a  considerable  difference,  for  at  least  two  reasons.  First,  when 
a  nested  logit  model  is  estimated  by  sequential  ML,  as  in  the  TER  Report,  changing  the  nesting 
structure  changes  both  which  variables  are  allowed  to  influence  angler's  choices  and  how  they 
exert  this  influence.  This,  in  turn,  affects  estimates  of  model  coefficients.  Second,  regardless  of 
whether  one  uses  sequential  ML  or  FIML,  the  nesting  structure  in  a  generalized  logit  model 
imposes  very  specific  assumptions  about  the  pattern  of  correlation  among  the  random  components 
in  the  underlying  RUM.  These  have  a  powerful  influence  on  predictions  of  behavior  and 
estimates  of  value.  Within  each  branch  of  a  nest  in  the  logit  model,  the  items  within  the  branch 
are  required  to  satisfy  a  property  known  as  Independence  of  Irrelevant  Alternatives  (IIA).  For  the 
tree  as  whole,  the  model  sUiicture  implies  a  degree  of  correlation  among  the  items  within  a 
branch,  but  not  between  items  in  one  branch  and  items  in  another.  If  the  assumed  nesting 
structure  is  incorrect  and  the  covariance  structure  among  the  stochastic  components  is  mis- 
specified,  this  causes  the  coefficient  estimates  to  be  biased.  Alvarez  and  Nagler  (1994)  have 
shown  that  the  bias  can  be  large,  and  can  affect  not  just  the  absolute  values  of  the  coefficient 
estimates  but  also  their  ratios. 
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Although  the  TER  research  team  model  the  choice  of  trip  length  in  the  bottom  nest  of 
their  tree,  they  do  so  in  a  peculiar  manner  which  is  undermined  by  their  approach  to  measuring 
travel  cost.  As  I  noted  earlier,  in  order  to  reduce  the  estimates  of  lost  consumer's  surplus,  they 
omit  several  components  of  travel  cost  that  most  other  researchers  would  have  included.  Several 
of  these  omitted  components  are  items  that  vary  with  trip  length  —  for  example,  costs  of  food 
and  lodging,  and  on-site  time.  Without  these  variables,  there  is  no  basis  for  modelling  the  choice 
of  trip  length  as  an  economic  decision.  There  is  no  point  in  even  collecting  data  on  trip  length, 
if  one  is  going  to  exclude  the  economic  variables  that  determine  it.  The  TER  research  team 
sought  to  finesse  this  by  including  dummy  variables  for  trip  length  in  the  equation  to  explain  to 
explain  trip  length.  Not  surprisingly,  these  variables  have  highly  significant  coefficients.  But, 
most  people  would  consider  regressing  a  variable  on  itself  to  be  a  dubious  way  of  "explaining" 
it. 


5.4  Model  Estimation 

Not  only  did  the  TER  research  team's  decision  to  combine  site  choice  with  choice  of  trip 
length  cause  their  model  to  be  mis-specified,  but  it  also  created  a  nightmare  for  them  in 
estimating  the  model  because  it  expanded  the  number  of  choice  items  at  the  bottom  of  the  tree 
fourfold,  from  97  to  388.  Because  they  could  not  cope  with  so  many  choices,  they  resorted  to 
a  technique  of  sampling  a  subset  of  the  alternatives.  The  TER  model  implies  that  anglers  visiting 
a  river  site  choose  among  236  (=  4x59)  alternative  site/length  combinations,  and  those  visiting 
a  lake  choose  among  152  (=  4x38)  alternative  site/length  combinations.  However,  when  the  TER 
research  team  actually  estimated  the  model,  they  assumed  that  anglers  visiting  a  river  site  choose 
among  only  25  alternatives,  and  similarly  for  anglers  visiting  a  lake.'^  This  certainly  reduced 
the  dimensionality  of  the  model  and  simplified  its  estimation.  This  approach  was  first  suggested 
by  Professor  McFadden.  It  involves  randomly  sampling  some  fraction  of  the  alternatives  —  the 
TER  research  team  settled  on  25  in  this  case,  amounting  to  1 1  %  of  the  alternatives  for  rivers,  and 
16%  for  lakes. 

McFadden  (1978)  shows  that,  if  the  items  satisfy  the  IIA  property,  the  estimates  obtained 
through  sampling  of  alternatives  are  consistent  estimates  of  the  true  coefficients.  However,  I  have 
three  reservations  about  the  use  of  sampling  in  this  case.  First,  appealing  to  the  asymptotic  of 
consistency  seems  inappropriate  where  there  are  such  tiny  samples  of  trips  ~  it  is  more  like 
wishful  thinking.  Second,  the  TER  research  team  implemented  the  sampling  incorrectly.  Instead 
of  sampling  randomly  without  replacement,  they  sampled  non-randomly  and  with  replacement. 
They  produced  a  set  of  25  alternatives  which  involves  duplications  of  the  same  choice  items 
within  the  set.  We  believe  that  this  was  unintentional,  and  it  may  have  resulted  from  carelessness 
or  haste  in  performing  the  final  model  runs.  Earlier  model  runs  had  used  a  different  version  of 
the  model  from  what  appears  in  the  TER  Report,  in  which  there  were  6  rather  than  4  trip  lengths, 
resulting  in  6x97  rather  than  4x97  alternatives.  It  seems  that  the  sampling  of  25  alternatives  was 


This  is  mentioned  nowhere  in  the  report;  it  just  shows  up  in  the  computer  code. 
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performed  at  that  stage  of  the  analysis.  When  the  change  was  made  to  4  trip  lengths,  the 
alternatives  selected  for  the  sample  were  not  appropriately  modified  —  the  top  three  trip  lengths 
were  collapsed  into  the  new  top  trip  length.  The  result  is  both  an  excess  proportion  of  the  25 
alternatives  in  that  length  category  and,  sometimes,  a  duplication  with  two  or  more  alternatives 
being  the  same  site/length  combination.  I  understand  that  Professor  McFadden's  result  on  the 
asymptotic  reliability  of  coefficient  estimates  under  sampling  does  not  hold  in  these 
circumstances. 

Moreover,  Professor  McFadden's  result  requires  the  multinomial  logit  structure  with  its 
IIA  property.  Thus,  the  EA  property  is  fundamental  to  both  the  specification  of  the  TER  model 
and  the  use  of  sampling  to  simphfy  its  estimation.  It  is  all  the  more  surprising  that  the  TER 
research  team  never  got  around  to  testing  whether  the  data  satisfy  this  property.  My  assistant 
Craig  Mohn  has  tested  for  IIA  among  the  river  and  lake  choices  at  the  bottom  of  the  tree  using 
the  Hausman-McFadden  (1984)  test  and  found  that  it  is  uniformly  rejected  in  both  cases. 
McFadden  (1984)  notes  that  the  test  is  not  very  powerful  "unless  deviations  from  the  multinomial 
structure  are  substantial."  That  assuredly  applies  to  the  TER  model. 

The  rejection  of  HA  implies  that  the  model  is  mis-specified.  This  can  be  a  symptom  of 
many  problems  ~  not  only  an  incorrect  nesting  structure  but  also  measurement  error  in  variables 
that  introduces  some  correlation  among  alternatives.  Both  are  likely  to  be  present  in  this  case. 
Whatever  the  cause,  the  consequence  of  the  failure  of  the  IIA  test  is  that  one  can  have  little 
confidence  in  the  coefficient  estimates  presented  in  the  TER  report. 

It  is  also  evident  that  the  coefficient  estimates  are  fragile  and  non-robust.  Table  12 
illustrates  this  by  comparing  several  of  the  coefficients  in  the  river  branch  at  the  bottom  of  the 
tree  when  one  uses  the  full  set  of  4*59  alternatives  rather  than  the  25  alternatives  employed  by 
the  TER  research  team.  This  is  the  estimation  that  they  wanted  to  perform  but  could  not  because 
it  was  too  large  for  them  to  handle.  Observe  how  substantially  the  coefficients  change.  Other 
evidence  of  fragility  is  provided  by  the  divergence  between  the  FEML  and  sequential  ML 
estimates  of  coefficients  in  the  TER  Report. 
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6.  Estimate  of  Recreation  Losses 

6.1  Fishing  Recreation  Loss 

The  TER  Report  assumes  away  the  loss  of  consumer's  surplus  for  nonresidents  who  fish 
in  Montana.  There  is  no  justification  for  this.  In  my  view,  it  is  clearly  an  important  and  large 
component  of  the  impact  of  hazardous  releases  on  fishing  in  the  Upper  Clark  Fork  Basin.  At  this 
point,  however,  I  have  not  had  time  to  form  an  estimate  of  it.  I  focus  instead  on  the  loss  to 
fishing  by  resident  anglers. 

The  TER  Report  presents  estimates  of  both  the  annual  damage  to  resident  anglers  and  the 
discounted  total  loss  over  the  period  1981  -  2000.  Here,  I  focus  on  the  estimate  of  annual 
damage.  The  TER  Report  presents  three  estimates.'^  The  base  estimate  is  $104,000  per  year, 
based  on  a  particular  scenario  for  the  effects  of  the  release  of  hazardous  substances  on  trout 
biomass  in  the  Upper  Clark  Fork  River  Basin  and  using  the  sequential  ML  model  coefficients. 
At  this  time,  I  am  not  in  a  position  to  form  an  opinion  regarding  the  effects  of  hazardous  releases 
on  the  aquatic  ecosystem  in  the  Basin.  For  the  sake  of  analysis  in  this  section,  I  take  the  TER 
assessment  of  these  effects  and  examine  what  monetary  value  should  be  placed  on  them. 
Moreover,  I  have  not  yet  had  the  time  to  replicate  the  calculation  of  lost  consumer's  surplus.  For 
now,  I  assume  that  the  baseline  estimate  was  calculated  correctly,  given  the  scenario  TER  chose 
to  employ.  The  TER  report  applies  two  cumulative  adjustments  to  this  baseline,  as  follows: 


TER  ADJUSTMENT 

ADJUSTMENT 
FACTOR 

ANNUAL 
DAMAGES 

Measurement  error  in  biomass 

X  1.19 

$130,500 

Upper  bound  on  95%  confidence  interval 

x2.07 

$411,500 

The  adjustment  for  error  in  measuring  biomass  is  based  on  the  Monte  Carlo  simulation  that  was 
discussed  above.  The  second  adjustment  is  intended  as  a  "conservative  upper-bound"  on  damages. 
However,  it  is  hardly  conservative,  because  it  is  based  on  estimate  of  variance  that  ignores  the 
imprecision  introduced  by  the  use  of  sample  weights.  Also,  it  is  not  clear  from  the  TER  Report 
whether  the  standard  errors  of  the  sequential  ML  coefficient  estimates  have  been  adjusted 
upwards  to  correct  for  the  fact  that  the  estimation  of  upper  levels  of  the  model  involve  estimated 
explanatory  variables. 


19 


It  also  presents  an  estimate  based  on  the  FTML  model.  I  discount  this  because  the  wrong  sign  of  coefficients 
in  that  model  highlights  the  havoc  that  mis-specification  can  wreak. 
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I  now  modify  these  adjustments  to  take  account  of  the  conclusions  I  have  reached  about 
the  surveys,  the  data,  and  the  model  structure  and  estimation  that  are  used  in  TER's  analysis.  I 
will  not  consider  a  95%  confidence  interval  upper  bound,  since  I  have  not  yet  had  time  to 
investigate  this. 

If  one  adopts  TER's  approach  to  modelling  recreation  choices  by  resident  anglers  but 
merely  corrects  for  the  gross  errors  in  their  analysis,  there  are  four  items  that  must  be  corrected: 
(i)  the  undercount  of  fishing  trips,  (ii)  the  bias  introduced  by  not  estimating  the  river  model  with 
the  full  set  of  site/length  alternatives  that  the  TER  research  team  postulates,  (iii)  the  errors  in 
measimng  travel  cost,  and  (iv)  the  error  in  measuring  the  biomass  variable.  If  one  assumed  that 
the  release  of  hazardous  substances  in  the  Basin  degraded  sites  from  a  value  of  1  in  the 
AESTHETICS  variable  to  a  value  of  0,  this  would  add  another  component  to  the  estimate  of  loss. 
Setting  that  aside,  the  damage  estimate  in  the  TER  Report  should  be  adjusted  as  follows: 


RECOMMENDED  ADJUSTMENT 

ADJUSTMENT 
FACTOR 

ANNUAL 
DAMAGES 

Raw  data  error:  undercount  of  trips 

X  2 

$208,000 

Estimation  error  from  not  using  full  set 
of  river  alternatives 

X  1.41 

$293,280 

Measurement  error,  travel  cost 

X  2 

$586,500 

Measurement  error,  biomass 

x2.4 

$1,407,700 

6.2  Non-Fishing  Recreation  Loss 

One  of  the  oddest  features  of  the  TER  study  is  that,  while  the  research  team  went  to  great 
lengths  to  collect  data  on  a  wide  variety  of  outdoor  recreation  activities  by  Montana  residents, 
including  several  that  are  likely  to  have  been  impacted  by  the  release  of  hazardous  substances 
in  the  Upper  Clark  Fork  Basin,  it  then  ignored  all  activities  other  than  fishing.  A  by-product  of 
doing  this  was  that  the  TER  Report  fails  to  present  any  estimate  of  recreation  damages  for 
activities  other  than  fishing.  There  can  be  no  good  justification  for  this  failure.  I  have  not  yet  had 
time  to  form  an  estimate  of  the  possible  damage  to  non-fishing  recreation  in  Montana  by  either 
residents  or  non-residents.  However,  it  would  clearly  be  significant. 
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Summation  in  Multi-Part  Policies:  An  Empirical  Test  of  Whether  'Too  Many  Proposals  Pass 
the  Benefit-Cost  Test'."  Department  of  Environmental  Studies,  University  of  California,  Davis, 
August,  1990. 

Carson,  Richard  T.;  Flores,  Nicholas;  Hanemann,  W.  Michael.  "On  the  Nature  of  Compensable  Value 
in  a  Natural  Resource  Damage  Assessment"  Natural  Resource  Damage  Assessment,  Inc.:  La 
Jolla,  California.  December,  1991. 
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Carson.  Richard  T.;  Conaway.  M.B.;  Hanemann,  W.M.;  Kopp.  R.J.;  Martin,  K.;  Mitchell, 
R.C.;  Presser.  S.;  Ruud.  P.A.  "Valuing  Winter  Visibility  Improvement  in  the  Grand 
Canyon,  Natural  Resource  Damage  Assessment,  Inc.:  La  Jolla,  California,  1992. 

KristrOm,  Bengt;  Nyquist,  Hans;  and  Hanemann,  W.  Michael.  "Dichotomous  Choice  Contingent 
Valuation  Experiment:  A  Mixed  Distribution  Approach."  Natural  Resource  Damage 
Assessment,  Inc.:  La  Jolla.  California.  1993. 

Hanemann,  W.  Michael.  "Projected  M&I  Demand  in  the  LADWP  Service  Area  1990-2010." 
Department  of  Agricultural  and  Resource  Economics,  Working  Paper  No.  644, 
University  of  California,  Berkeley.  February,  1993. 

Albers.  Heidi  J.;  Fisher,  Anthony  C;  and  Hanemann,  W.  Michael.  "Valuation  and 

Management  of  Tropical  Forests:  A  Theoretical  and  Empirical  Analysis."   Department 
of  Agricultural  and  Resource  Economics,  Working  Paper  No.  681,  University  of 
California,  Beri^eley.  July,  1993. 

Chesnutt,  Thomas  W.;  Bamezai,  Anil;  and  Hanemann,  W.  Michael.  "Revenue  Instability 
Induced  by  Conservation  Rate  Structures:  An  Empirical  Investigation  of  Coping 
Strategies."   Department  of  Agricultural  and  Resource  Economics,  Working  Paper 
No.  695.  University  of  California,  Beriieley.  November,  1993. 


Hanemann,  W.  Michael.  "Contingent  Valuation  and  Economics."   Department  of  Agricultural 
and  Resource  Economics,  Working  Paper  No.  697.  University  of  California.  Berkeley, 
1994. 

Hanemann,  W.  Michael.  "Marginal  Cost  Pricing  and  the  New  LADWP  Water  Rates." 
Department  of  Agricultural  and  Resource  Economics,  Working  Paper  No.  682, 
University  of  California.  Berkeley,  1994. 


Research  Reports 

Hanemann,  W.  Michael;  and  Binkley,  C.  The  Recreation  Benefits  of  Water  Quality  Improvements: 

Analysis  of  Day  Trips  in  an  Urban  Setting.  EPA-600/5-78-010.  Report  submitted  to  the  U.  S. 
Environmental  Protection  Agency  by  Urban  Systems  Research  and  Engineering,  Inc. 
Washington,  D.  C:  U.  S.  Environmental  Protection  Agency,  1978. 

King,  Diana;  Hanemann,  W.  Michael;  and  Nimmons,  John.  The  Role  of  Gas  and  Electric  Utilities  in 
Direct  Applications  of  Geothermal  Resources.  Earl  Warren  Legal  Institute,  University  of 
Califoniia.  Berkeley,  March,  1980. 

Hanemann,  W.  Michael  (contributor).  Market  Shares  Estimation  Task  Force  Report:  Projections  for 
Hydrothermal  Direct-Heat  Systems  Market  Shares.  U.  S.  Department  of  Energy,  December. 
1980. 

Hanemann.  W.  Michael.  Welfare  Evaluations  with  Simulated  Hypothetical  Market  Data: 
Bishop  and  Heberlein  Revisited.  1983. 
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Fairfax.  Sally  K.;  and  Hanemann,  W.  Michael,  eds.  Emerging  Research  Needs  and  Priorities 

Concerning  Minerals  Development  on  National  Forest  Lands.  I^cpartnicnt  of  Agricultural  and 
Resource  Economics,  University  of  California.  Berkeley,  March,  1983. 

Stem.  Martin  W.;  Hanemann,  W.  Michael;  and  Eckhouse,  Kathy.  A  Case  Study  of  Energy  and  Process 
Substitution  for  Frozen  Broccoli:  Geothermal  Energy  and  the  Retortahle  Pouch.  Department 
of  Agricultural  and  Resource  Economics,  University  of  California.  Berkeley,  August,  1983. 

Hanemann.  Michael;  Strand,  Ivar,  and  Wegge.  Thomas.   Albacore  Sportfishing  Economic  Study. 

Prepared  for  the  National  Marine  Fisheries  Service.  Sacramento,  California:  Jones  and  Stokes 
Associates.  Inc..  May.  1986. 

Hanemann.  W.  Michael.  Economic  Valuation  of  Changes  in  the  Catch  of  Sacramento  River  Chinook 
Salmon.  Department  of  Agricultural  and  Resource  Economics,  University  of  California. 
Beiiieley,  October,  1986. 

Hall,  C.  H.;  Lichtenberg,  E.  R.;  Patton,  S.  E.;  Layton,  D.  W.;  Hanemann,  W.  M.;  and  Stuermer.  D.  H. 
Costs  of  Alternative  Treatment  Technologies  for  1 ,2-Dibromo-3-Chloropropane  (DBCP)  in 
California  Groundwater  Supplies.  Livermore,  California:  Lawrence  Livermore  National 
Laboratory,  May,  1987. 

Carson.  Richard  T.;  Hanemann,  W.  Michael;  and  Wegge,  Thomas  C.  Final  Report:  Juneau  Area  Sport 
Fishing  Economic  Study.  Prepared  for  the  Alaska  Department  of  Fish  and  Game.  Jones  and 
Stokes  Associates,  Sacramento,  California,  December,  1987. 

.  Final  Report:  Southcentral  Alaska  Sport  Fishing  Economic  Study.  Prepared  for  the  Alaska 

Department  of  Fish  and  Game.  Jones  &  Stokes  Associates,  Sacramento,  California,  December, 
1987. 

Hanemann,  W.  M.;  and  Dale,  L.  Reasonable  Municipal  and  Industrial  Use  in  2010  by  SWP 

Contractors  in  Southern  California  and  Reasonable  Municipal  and  Industrial  Use  in  2010  in 
the  Central  Coast  and  San  Francisco  Bay  HSAs.  Consultant's  report  submitted  to  the  State 
Water  Resources  Control  Board,  Berkeley,  1988. 

Dale,  L.;  and  Hanemann,  W.  M.  The  Cost  of  Re-Operating  Ncm'  Don  Pedro  Resenoir  to  Increase 

Spring  Flow  to  the  San  Joaquin  River.  Department  of  Agricultural  and  Resource  Economics, 
University  of  California.  Berkeley.  June.  1990. 

Dale,  L.;  and  Hanemann,  W.  M.  Sizing  of  A  Groundwater  Bank.  Department  of  Agricultural  and 
Resource  Economics,  University  of  California.  Bericeley,  June,  1990. 

Jones  &  Stokes  Associates,  Inc.  Environmental  Benefits  Study  of  San  Joaquin  Valley's  Fish  and 

Wildlife  Resources.  (JSA  87-150)  Sacramento,  California.  Prepared  by  J.  B.  Loomis,  W.  M. 
Hanemann,  and  T.  C.  Wegge,  September,  1990. 

Carson,  Richard  T.;  Hanemann,  W.  Michael;  Kopp.  Rayinond.  A  Nested  Logit  Model  of 
Recreational  Fishing  Demand  in  Alaska,  June,  1989. 

Jones  &  Stokes  Associates,  Inc.  Southeast  Alaska  Sport  Fishing  Economic  Study  Final  Research  Repwrt 
(JSA  88-028)  Sacramento,  California.  December  1991. 
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Carson,  Richard  T.;  Conaway,  Michael  B.;  Hanemann  W.  Michael;  Kopp,  R.J.;  Mitchell.  R.C.; 
Presser,  S.;  Ruud,  P.A.  Comments  on  the  Benefit  Analysis  in  the  U.S.  Environmental 
Protection  Agency's  Proposed  Navajo  Generating  Station  BART  Action  Decision 
Focus.  1991. 

Neal  MacDougall,  Hanemann.  W.  Michael,  and  Zilberman.  David.  The  Economics  of  Agricultural 
Drainage  Report  submitted  to  the  Central  Valley  Regional  Water  Quality  Control  Board, 
Standard  Agreement  No.  0-132-150-0.  June  1992. 

Hanemann.  W.  Michael.  Comment  submitted  to  the  Office  of  Environmental  Affairs, 

Department  of  Interior,  in  response  to  the  Notice  of  Proposed  Rulemaking  (58  FR 
39328)  Concerning  Natural  Resource  Damage  assessments,  1993. 

Dumas,  Christopher  F.;  and  Hanemann,  W.  Michael.  Economic  Benefits  of  the  United  States 
Environmental  Protection  Agency's  Proposed  Bay-Delta  Standards.  Report  for  United 
States  Environmental  Protection  Agency,  Water  Management  Division,  San  Francisco, 
CA.  1994. 

Hanemann,  W.  Michael;  McCann.  Richard.  "Economic  Impacts  on  the  Northern  California 
Hydropower  System"  in  Integrated  Modeling  of  Drought  and  Global  Warming. 
Impacts  on  Selected  California  Resources.   National  Institute  for  Global 
Environmental  Change,  University  of  California,  Davis.  Pages  55-68.  March,  1993. 

Dumas,  Christopher,  and  Hanemann,  W.  Michael.  "Simulating  Impacts  on  the  Sacramento 

River  Fall  Run  Chinook  Salmon  Population"  in  Integrated  Modeling  of  Drought  and 
Global  Warming,  Impacts  on  Selected  California  Resources.   National  Instimte  for 
Global  Environmental  Change,  University  of  California,  Davis.  Pages  69-95.  March. 
1993. 

Hanemann.  W.  Michael.  "Economic  Impact  on  Ocean  Fishing"  in  Integrated  Modeling  of 
Drought  and  Global  Warming,  Impacts  on  Selected  California  Resources.  National 
Institute  for  Global  Environmental  Change,  University  of  California.  Davis.  Pages 
107-112.  March.  1993. 

Green,  Gareth;  Hanemann,  W.  Michael;  Helfand,  Gloria;  and  Larrivee,  Francois.  "Economic 
Valuation  of  Changes  in  the  Quality  of  Life."  Report  for  the  California  Department 
of  Health  Services,  1994. 

Mitchell,  David  L;  and  Hanemann.  W.  Michael.  "Setting  Urban  Water  Rates  for  Efficiency 
and  Conservation:  A  Discussion  of  Issues."  Report  for  the  California  Urban  Water 
Conservation  Council,  September-October  1994. 
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Recent  Presentations 


"Modeling  Recreation  Demand  in  a  Multiple  Site  Framework."  Presented  at  the  Association  of 
Environmental  and  Resource  Economists  Workshop  on  Recreation  Demand  Modeling. 
Boulder.  Colorado.  May  16  and  17,  1985. 

"Valuing  Genetic  Resources."  Presented  at  the  University  of  Washington  Conference  on  the 
Conservation  of  Genetic  Resources.  Seattle,  Washington.  July  17-21.  1985. 

"The  Evaluation  of  Environmental  Improvements:  Whose  Benefits  Get  Counted?"  (coauthored  with 
Nancy  Bockstael  and  Ivar  Strand).  Presented  at  the  Western  Agricultural  Economics 
Association  Annual  Meeting.  Saskatoon.  Canada,  July  7-9.  1985. 

"Economic  Techniques  for  Measuring  Environmental  Damages."  Invited  lecture  at  the  California 

Department  of  Fish  and  Game  Conference  on  Environmental  Damage  Assessment.  Monterey. 
California.  October  9.  1985. 

"Environmental  Protection:  Recovery.  D>Tiamics.  and  Benefit  Evaluation"  (coauthored  with 

Anthony  C.  Fisher).  Presented  at  the  Conference  on  Man's  Role  in  Changing  the  Global 
Environment,  Venice,  Italy,  October  21-26,  1985. 

"The  Hysteresis  Phenomenon  and  Benefit  Evaluation  for  Pollution  Control  in  Aquatic  Ecosystems" 
(coauthored  with  Anthony  C.  Fisher).  Presented  at  the  American  Economics  Association 
Annual  Meeting,  New  York,  December  27-29,  1985. 

"Adjusting  for  Sampling  Effects  in  Demand  Analysis  with  Single  and  Multiple  Sites."  Invited  lectures 
at  the  W-133  Annual  Meeting,  San  Diego,  California,  February  19-27.  1986. 

"Implications  of  Biometrics  for  the  Design  of  Dichotomous  Choice  Contingent  Valuation 

Experiments."  Invited  lectures  at  the  W-133  Annual  Meeting,  San  Diego.  California, 
February  19-27,  1986. 

"Economic  Analysis  of  Drainage  Problems."  Presented  at  the  University  of  California  Conference  on 
Decision  Criteria  for  Residuals  Management  in  Agriculture,  Davis,  California,  April  17 
and  18,  1986. 

Panelist,  Resources  for  the  Future  Conference  on  Agriculture  and  the  Environment,  Washington,  D.  C, 
April  20-22,  1986. 

"Pollution  Control  in  a  Dynamic  Setting."  Presented  at  the  Association  of  Environmental  and  Resource 
Economists  Woricshop  on  Marine  Pollution  and  the  Environmental  Damage  Assessment, 
Kingston.  Rhode  Island.  June  5-7.  1986. 

Panelist  National  Marine  Fisheries  Service,  Workshop  on  Recreational  Fishing  Data  Needs,  Portland, 
Oregon.  June  18.  1986. 

"On  the  Integration  of  Some  Common  Demand  Systems"  (coauthored  with  Jeffrey  La  France). 

Presented  at  the  Western  Economics  Association  Annual  Meeting.  San  Francisco,  California, 
July  1-3.  1986. 
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Chair,  Economics  Panel.  National  Academy  of  Science/Smithsonian  Institution,  National  Forum  on 
Biodiversity,  Washington,  D.  C,  September  21-24.  1986. 

"WTP  and  WTA  as  Benefit  Measures — How  Do  They  Differ?  How  Can  They  Be  Mea.sured?" 
Presented  at  the  Workshop  on  Economic  Nonmarket  Valuation,  Monterey,  California, 
November  14  and  15,  1986. 

"Weak  Separability,  Partial  Demand  System,  and  Exact  Consumer's  Surplus"  (coauthored  with 

Edward  R.  Morey).  Presented  at  the  American  Economics  Association  Annual  Meeting,  New 
Orleans,  Louisiana.  December  27-29,  1986. 

"Some  Simple  Techniques  for  Valuing  Improvements  in  Sports  Fishing."  Presented  at  W-133  Annual 
Meeting,  El  Paso.  Texas.  February  11-13,  1987. 

"Institutional  Problems  of  Production  Agriculture  Related  to  the  Regulatory  F*rocess."  Presented  at 
Workshop  on  Water  Quality  in  Agriculture,  California  State  University,  Fresno,  March  20, 
1987. 

"Discrete  Response  Contingent  Valuation  Experiments."  Presented  at  the  Economics  Department, 
Vanderbilt  University,  April  3,  1987. 

"Welfare  Evaluations  in  Discrete/Continuous  Choice  Models."  Invited  lectures  at  the  Economics 

Department  and  College  of  Natural  Resources,  University  of  Michigan,  Ann  Arbor,  April  20 
and  21.  1987. 

"Current  Water  Policy  Issues  in  California."  Invited  lectures  at  the  Economics  Department  and  College 
of  Natural  Resources.  University  of  Michigan.  Ann  Arbor,  April  20  and  21.  1987. 

"The  Dual  Structure  of  Incomplete  Demand  Systems"  (coauthored  with  Jeffrey  T.  La  France). 

Presented  at  the  Operations  Research  Society  Meetings.  New  Orleans,  Louisiana,  May  4-6, 
1987. 

"The  Structure  of  Economic  Benefit  Measures."  Invited  lecture  at  the  Economics  Department, 
University  of  California  at  San  Diego,  June  10,  1987. 

"The  Methodology  of  Discrete  Response  Contingent  Valuation  Experiments."  Presented  at  the  Western 
Economics  Association  Meetings,  Vancouver,  British  Columbia,  July  9,  1987. 

Invited  panelist.  Fisheries  Canada  Workshop  on  Recreational  Fisheries  Evaluation,  Vancouver. 
British  Columbia,  July  10,  1987. 

"Biological  Uncertainty  and  the  Analysis  of  Pollution  Control  Policy."  Presented  at  International 

Society  for  Ecological  Modeling  Symposium  on  Ecological  Economics.  Ohio  State  University, 
August  12.  1987. 

"Economic  Valuation  of  Natural  Environments."  Invited  course  of  lectures.  Economics  Department, 
University  of  Oslo.  Norway.  September  3-15.  1987. 

"A  Generalized  Logit  Model  of  Sportfishing  in  Alaska  with  Economic  Welfare  Measures."  (coauthored 
with  Richard  T.  Carson).  Presented  at  the  Department  of  Forest  Economics,  Swedish 
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University  of  Agricultural  Sciences,  Umea,  September  22.  1987,  and  Stockholm  School  of 
Economics.  September  26,  1987. 

"Beyond  Contingent  Valuation:  Deriving  Environmental  BenefiLs  from  Hypothetical  Behavior  Data." 

Presented  at  the  American  Public  Policy  Association  Meeting,  Washington.  D.  C.  October  29. 
1987. 

"Economics  of  Option  Value."  Presented  at  Annual  Meeting  of  Society  for  Economic  Dynamics  and 
Control.  Phoenix.  March  8.  1988. 

Invited  panelist.  "Review  of  Workshop."  Association  of  Environmental  &  Resource  Economists 
Workshop  on  Sportfisheries  Valuation.  Seattle,  June  23-24.  1988. 

Invited  discussant.  Resources  for  the  Future  Conference  on  Natural  Resources  Damage  Assessment 
Washington.  D.  C.  June  15-16,  1988. 

Invited  discussant.  Western  Economic  Association  Annual  Meeting,  Session  on  Issues  in  Fisheries 
Economics,  Los  Angeles,  July  3,  1988. 

Invited  discussant.  University  of  Maryland  Symposium  on  Commercial  Agriculture  and  Resource 
Policy.  Baltimore.  May  4-5,  1989. 

"The  Supply  Effects  of  Meeting  Water  Quality  Standards."  Invited  paper  presented  at  Western 
Economic  Association  Annual  Meeting,  South  Lake  Tahoe,  June  20,  1989. 

"A  Nested  Logit  Model  of  Recreational  Fishing  Demand  in  Alaska"  (coauthored  with  Richard  T. 

Carson  and  Thomas  Wegge).  Presented  at  Western  Economic  Association  Annual  Meeting. 
South  Lake  Tahoe.  June  22,  1989. 

"Zonal  Travel  Demand  Modelling  Revisited."  Invited  paper  presented  at  Western  Agricultural 
Economics  Association  Annual  Meeting.  Coeur  d'Alene,  July  12,  1989. 

"The  Role  of  Water  Conservation  in  Agricultural  Drainage  Control."  (coauthored  with  Erik 

Lichtenberg  and  David  Zilberman).  Presented  at  American  Agricultural  Economics  Association 
Aruiual  Meeting,  Baton  Rouge,  August  1,  1989. 

"Myopia  and  the  Management  of  Stochastic  Natural  Resources  in  the  Face  of  Alternative  Hypotheses." 
Presented  at  Oslo  Center  for  Applied  Research  Conference  on  Stochastic  Models  and  Option 
Prices:  Applications  to  Resources,  Environment  and  Investment  Problems,  Loen,  Norway, 
August  28-30.  1989. 

Invited  panelist.  "Review  of  Workshop."  Alaska  Department  of  Fish  &  Game  Wildlife  Valuation 
Workshop.  Denali/Anchorage,  September  9-15.  1989. 

"Multicomponent  Valuation:  Valuing  the  Natural  Environment  as  a  Multi-Dimensional  Commodity." 
Presented  at  seminars  in  the  Department  of  Economics.  University  of  Illinois  at  Urbana, 
November  15,  1989,  and  the  Rand  Corporation,  Santa  Monica,  December  1,  1989. 

"Review  of  SOS/T  Report  27:  Methods  for  Valuing  Acidic  Deposition  and  Air  Pollution  Effects." 
Invited  paper  presented  at  NAPAP  International  Conference-Acidic  Deposition:  Slate  of 
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Science  and  Technology.  Hilton  Head  Island.  February  14.  1990. 

Invited  participant.  Rockefeller  Foundation  Seminar  on  Advances  in  the  Economics  of  Valuation  and 
Environmental  Control.  Bellagio,  Italy,  April  9-11.  1990. 

"The  Economics  of  Glen  Canyon  Dam."  Presented  at  National  Academy  of  Sciences  Conference  on 
the  Colorado  River.  Santa  Fe.  New  Mexico.  May  24-26,  1990. 

"Empirical  Analysis  in  the  Use  of  Contingent  Valuation."  Presented  at  American  Agricultural 

Economics  Association  Learning  Worlcshop,  "Nonmaricet  Valuation:  Extending  the  Frontiers 
and  New  Applications."  VarKOuver.  August  7,  1990. 

"Valuation  of  Tropical  Rain  Forests"  (coauthored  with  Anthony  C.  Fisher).  Presented  at  United 

Nations  University  WIDER  Conference  on  The  Environment  and  Emerging  Developing  Issues, 
Helsinki,  September  3-9.  1990.  and  the  American  Economics  Association  Annual  Meeting. 
Washington,  D.C.  December  27-29.  1990. 

"Economic  Aspects  of  Natural  Resource  Damage  Assessment."  Presented  at  National  Association  of 
Attorneys  General  Advanced  CERCLA/RCRA  Enforcement  Seminar.  Washington.  D.C. 
September  24-26,  1990. 

"The  Theory  of  Measuring  the  Benefits  of  Quality  Change."  Presented  at  the  W-133  Annual  Meeting, 
Monterey,  CA  February  27,  1991. 

Panelist,  Colloquium  on  Water  Quality  Modelling  -  Science,  Environmental  Management  and  Policy. 
Presented  by  the  San  Francisco  Estuary  Project  at  U.C.  Bericeley  on  April  9,  1991. 

"The  Economics  of  Loss  Aversion."  Presented  at  the  Department  of  Economics  &  Business,  North 
Carolina  State  University,  Raleigh,  May  16,  1991 

"The  Economics  of  Environmental  Regulation."  Presented  at  1991  Program  of  the  Tahoe-Baikal 
Institute,  Tahoe  City,  August  2,  1991. 

Discussant,  Peder  Sather  Symposium  on  Global  Qimaie  Change:  European  and  American  Policy 
Responses.  UC  Berkeley,  October  16-18,  1991. 

Invited  Participant.  NOAA/EPA  Workshop  on  Environmental  Benefits  Transfer  and  Nonmarket  Values 
Database.  Washington,  D.C.  November  14.  1991. 

"The  Benefits  and  Costs  of  Irrigation  Delivery  Rexibility."  (co-authored  with  Larry  Dale)  Presented  at 
US  Committee  of  Irrigation  &  Drainage  Biennial  Meeting,  San  Francisco,  December  10,  1991. 

"On  the  Nature  of  Compensable  Value  in  a  Natural  Resource  Damage  Assessment."  (coauthored  with 
Richard  T.  Carson  and  Nicholas  E.  Flores)  I^esented  at  the  American  Economic  Association 
Annual  Meeting,  New  Orleans,  January  3,  1992. 

"The  Meaning  of  Existence  Value."  Presented  at  the  W-133  Annual  Meeting,  South  Lake  Tahoe, 
Febniary  27,  1992 

"Assessing  Qimate  Change  Risks:  Valuation  of  Effects."  (co-authored  with  Anthony  C  Fisher) 
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Presented  at  the  Resources  for  the  Future,  Inc.  workshop  on  "Assessing  Qimate  Change 
Risks".  Washington.  DC.  March  22-23,  1992. 

"Survey  Data  Collection:  Detecting  and  Correcting  for  Biases  in  Responses  to  Mail  and  Telephone 

Surveys."  (co-authored  with  Barbara  J.  Kanninen  and  David  J.  Chapman)  Presented  at  the  US 
Bureau  of  the  Census  1992  Aitnual  Research  Conference,  Arlington,  VA  March  22-25.  1992. 

"Critique  of  the  Exxon  Papers  on  CV."  (co-authored  with  Richard  Carson).  Presented  at  the  AAEA 
Annual  Meeting.  Baltimore  MD.  August  10,  1992. 

"Using  Economic  Incentives  to  Regulate  Pollution:  What  Do  Economists  Know?"  Invited  keynote 
address  at  Workshop  on  Environmental  Systems  and  Economics,  Society  for  Mathematics, 
Economics  &  Operations  Research  17th  Symposium  on  Operations  Research,  Universitat  der 
Bundeswehr,  Hamburg,  Germany.  August  25-26.  1992.  Also  presented  at  USDA-ERS  Staff 
Seminar.  Washington,  DC.  December  17th.  1992. 

"Recent  Experience  with  Natural  Resource  Damage  Assessment"  and  "Needs  for  Ongoing  Model 
Development."  hivited  presentations  at  North  Carolina  State  University  Natural  Resources 
Damage  Assessment  Workshop,  Raleigh.  NC.  September  18.  1992. 

"Lessons  from  the  Natural  Resources  Damage  Assessment."   presentation  at  North  Carolina 
State  University.  Resources  and  Environmental  Economics  Program,  Workshop  1992. 

"Valuing  Effects  of  Change  in  Harvest:  Marine  Recreational  Fisheries"  and  "Implementing  Valuation 
Estimates  in  Allocation  Decisions."  Invited  presentations  at  North  Carolina  Marine  Fisheries 
Economics  Summit,  Beaufort.  NC.  September  19,  1992. 

"The  Valuation  of  Natural  Resources:  The  Aftermath  of  the  Exxon  Valdez."  Invited  Lecture  in 

Distinguished  Speaker  Series,  University  of  Michigan  School  of  Natural  Resources  and  the 
Environment,  Ann  Arbor  MI,  October  26,  1992. 

Member,  Panel  on  "Economic  Benefits  of  Environmental  Protection:  How  Can  They  Be  Quantified?", 
Conference  on  California  Water  Policy:  Resolving  Critical  Controversies.  Los  Angeles, 
November  20-21.  1992. 

Invited  Participant,  EPRI/NOAA  Workshop  on  Coastal  Impacts  of  Climate  Change,  Washington  DC, 
December  15-16,  1992. 

"Emissions  Taxes  and  Markets:  What  Do  We  Really  Know?"  Seminar  presentation,  USDA  Economic 
Research  Service,  Washington  DC,  December  17,  1992 

"Non-Market  Valuation  Using  Contingent  Behavior:  Model  Specification  and  Consistency  Tests."  (co- 
authored  with  David  Chapman  and  Barbara  Kanninen)  Presented  at  the  American  Economic 
Association  Annual  Meeting,  Anaheim  CA,  January  6,  1993 

"Correlated  Discrete-Response  Contingent  Valuation."  (co-authored  with  David  Chapman)  Presented  at 
the  W-133  Annual  MeeUng,  Santa  Fe  NM,  March  15.  1993. 

"Current  Issues  in  Dicholomous  Choice  Contingent  Valuation  Experiments."  (co-authored  with  Bengt 
KristrOm).  Presented  at  the  Stockholm  School  of  Economics  Conference  on  Environmental 
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Economics.  Ulvon,  Sweden,  June  12,  1993. 

"Financial  Incentives  for  Conservation."  Presentation  to  POWER  Conference  on  Restoration 
Partnerships  Under  the  New  Water  Law,  Oaidand  CA,  June  18,  1993. 

"Marginal  Cost  Pricing  and  the  New  LADWP  Water  Rates."  Presented  at  the  Western 

Economics  Association  Annual  Meeting.  South  Lake  Tahoe  NV,  June  21-24,  1993. 

"Natural  Resource  Damage  Assessment:  Economic  Implications  for  Fisheries  Management." 
(co-authored  with  Ivar  Strand).  Presented  at  the  Agricultural  Economics  Association 
Annual  Meeting.  Orlando  FL,  August  2,  1993. 

"Water  Markets  -  A  Skeptical  View."  Presentation  to  UC  Berkeley  Conference  on  Regional  Water 
Management,  Beilceley  Marina  Marriott.  October  16,  1993. 

"The  Economics  of  Mono  Lake."  Presented  at  Stanford  University  Environmental  Economics  Seminar, 
November  9,  1993. 

"An  Appraisal  of  Contingent  Valuation  Techniques."  MIT  Energy  Policy  Workshop,  Cambridge,  MA 
November  18,  1993. 

"Pricing  Water."  Presentation  to  University  of  Washington  Law  School  Conference  on  Water  Rights  in 
the  21st  Century,  Olympia  WA.  January  29,  1994. 

"Learning  to  Live  with  Stated  Preference."  Invited  presentation,  session  on  Valuing  the  Environment: 
Where  Do  We  Stand?  American  Association  for  the  Advancement  of  Science  Annual  Meeting, 
San  Francisco,  CA.  February  19.  1994. 

"What  are  the  Economic  Costs  of  Biodiversity  Conservation  and  Who  Should  Bear  Them?" 

Presented  at  the  First  North  American  Conference  on  Social  and  Economic  Incentives 
for  the  Conservation  of  Biodiversity  and  Biological  Resources.   Mexico  City,  March 
9.  1994. 

"Problems  of  Applying  Contingent  Valuation  Methods."  Invited  presentation  at  Conference  on 
Environmental  Valuation  in  Context  organized  for  the  UK  Deparmient  of  the 
Environment  by  the  University  of  Newcastle  Centre  for  Rural  Economy,  Durham 
England.  March  25,  1994. 

"Referendum  Contingent  Valuation:  How  Many  Bounds  Are  Enough?,"  presented  with  Joseph 
Cooper  at  the  annual  AAEA  meeting.  San  Diego,  Calif.,  August,  1994. 

"A  Qear  Water  Policy  for  the  Delta  or  Just  More  Mucking  About?"  presented  at  the 
Sacramento  Economics  Roundtable.  Sacramento,  Calif.,  May  24,  1995. 

"Nonmarket  Valuation  under  Preference  Uncertainty:  Econometric  Models  and  Estimation," 
paper  presented  with  Bengt  KristrOm,  and  Chuan-Zhong  Li  at  the  Meeting  of  the 
European  Association  of  Environmental  Economics,  June.  1995. 
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